Game-Decompilation/botw
1
0
Fork 0
mirror of https://github.com/zeldaret/botw.git synced 2024-11-26 22:50:34 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Link to new documentation website

Browse Source
This commit is contained in:
Léo Lam 2022-10-19 15:44:21 +02:00
parent 2f62d26424
commit 1d7ce82908
No known key found for this signature in database
GPG Key ID: 0DF30F9081000741
3 changed files with 2 additions and 1256 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

730
Cheatsheet.md
View File

@ -1,730 +0,0 @@
# Breath of the Wild Decompilation Cheatsheet
## Things to try when a function has major differences
The following actions should help when basic blocks are in the wrong order, or when there are weird issues that involve comparisons or conditionals:
* Invert conditionals.
* Introduce inline functions. Or manually inline code.
* Add or eliminate return statements.
* Duplicate or deduplicate code.
* Independent memory loads/stores can be reordered, so you might need to reorder statements or conditions in the source code. For example, `if (x || y)` might have to be written as `if (y || x)`.
* Turn if/else into ternaries and vice versa. This doesn't always make a difference, though.
* uintptr_t do not always produce the same code as pointers, even for simple operations such as comparisons.
* Loops:
* Index-based loops
* u32 / s32 can make a difference, in particular for loop unrolling.
* `<` vs `!=` can affect codegen. If the trip count is known at compile-time, Clang will usually change `<` into `!=`, but you should still try `<` first.
* In some rare cases, Nintendo will use `!=` instead of `<`.
* Iterator loops
* Since we are targeting C++17, the preferred way to iterate over a container is to use a range-based for loop (e.g. `for (auto x : array)`).
* Sometimes, making the iterator appear explicitly is required to match the original code. Example: `for (auto it = array.begin(), end = array.end(); it != end; ++it)`
* In some rare cases, the end iterator is not kept in a variable, and instead it's recalculated at the end of each iteration. Example: `for (auto it = array.begin(); it != array.end(); ++it)`
* Sometimes it is possible to use `<algorithm>` functions (e.g. std::for_each, std::all_of, etc.) for simpler loops.
* And in some very rare cases (when dealing with EventFlow for example) it is sometimes *required* to use `<algorithm>` to match.
## Minor differences
* Incorrect comparison flags (getting >= instead of > for example)
* Invert conditionals.
* For integers: make sure you are using the correct signedness.
* For example, HI means that you should be using an unsigned integer.
* For floating-point, keep in mind that `x > 5.0` and `!(x <= 5.0)` are not equivalent because of NaN. This can reveal how an if/else statement is supposed to be written.
* Swapped CSEL operands
* Invert conditionals. (For ternaries, also swap the ? and : operands, obviously.)
* In some rare cases, `ptr == nullptr` and `!ptr` do not generate the same code.
* Extraneous function prologue/epilogue: this can happen when returning references. Change the return type to a pointer.
## sinit / static initializer / cxa_atexit
* If the second argument of a `_cxa_atexit` call is nullptr and the destructor is a nullsub, the object in question is likely a C-style array (not a std::array or a sead::SafeArray).
## Inline functions
This section lists some inline functions that are often used throughout the codebase.
### sead
sead is Nintendo's C++ standard library. It provides basic data structures such as strings and tree maps and many other essential components (e.g. threads, critical sections, file IO, etc.)
#### Strings
##### `sead::SafeString` constructor
```cpp
x.vptr = &sead::SafeString::vt;
x.cstr = "some string here";
```
⬇️
```cpp
sead::SafeString x = "some string here";
```
Note that the SafeString constructor is also implicitly called whenever a string literal is converted into an sead::SafeString (because it was passed as an argument to a function that expects an sead::SafeString for example).
---
##### `sead::FixedSafeString<N>` constructor
A `sead::FixedSafeString<N>` is a fixed-length SafeString. It derives from sead::BufferedSafeString (which derives from sead::SafeString) and contains a char[N] buffer right after the length.
Note: the field assignments may be in a different order.
```cpp
x._.cstr = (char*)&xxx; // some buffer right after `x`
x._.vptr = &`vtable for'sead::BufferedSafeStringBase<char>;
x.length = N; // where N is a number
sead::BufferedSafeStringBase<char>::assureTerminationImpl_(&x);
*x._.cstr = sead::SafeStringBase<char>::cNullChar;
x._.vptr = ...;
```
⬇️
```cpp
sead::FixedSafeString<N> x;
```
---
##### sead::SafeString::cstr
sead::SafeString::cstr returns a `const char*`, like std::string::c_str. You can expect it to be called whenever a SafeString needs to be passed to a function that takes a C-style string (`const char*`).
```cpp
string.vptr->assureTermination(...);
const char* ptr = string.cstr;
// do stuff with string.cstr
// note that the variable may not exist in the pseudocode
```
⬇️
```cpp
const char* ptr = string.cstr();
```
---
##### sead::SafeString::calcLength
```cpp
x.vptr->assureTermination(&x);
v12 = x.cstr;
str_length = 0LL;
v14 = (signed __int64)(v12 + 1);
while ( v12[str_length] != sead::SafeStringBase<char>::cNullChar )
{
if ( *(unsigned __int8 *)(v14 + str_length) == (unsigned __int8)sead::SafeStringBase<char>::cNullChar )
{
LODWORD(str_length) = str_length + 1;
break;
}
if ( *(unsigned __int8 *)(v14 + str_length + 1) == (unsigned __int8)sead::SafeStringBase<char>::cNullChar )
{
LODWORD(str_length) = str_length + 2;
break;
}
v15 = str_length + 2;
str_length += 3LL;
if ( v15 >= 0x80000 )
{
LODWORD(str_length) = 0;
break;
}
}
```
⬇️
```cpp
s32 str_length = x.calcLength();
```
Note that this function is commonly called from other sead::SafeString inline functions.
---
##### sead::BufferedSafeString::copy
```cpp
dest_cstr = dest_safestring.cstr;
source_safestring.vptr->assureTermination(&source_safestring);
source_cstr = source_safestring.cstr;
if ( dest_cstr != source_cstr )
{
source_safestring.vptr->assureTermination(&source_safestring);
v6 = 0LL;
v7 = (signed __int64)(source_safestring.cstr + 1);
while ( source_safestring.cstr[v6] != sead::SafeStringBase<char>::cNullChar )
{
if ( *(unsigned __int8 *)(v7 + v6) == (unsigned __int8)sead::SafeStringBase<char>::cNullChar )
{
LODWORD(v6) = v6 + 1;
break;
}
if ( *(unsigned __int8 *)(v7 + v6 + 1) == (unsigned __int8)sead::SafeStringBase<char>::cNullChar )
{
LODWORD(v6) = v6 + 2;
break;
}
v8 = v6 + 2;
v6 += 3LL;
if ( v8 >= 0x80000 )
{
LODWORD(v6) = 0;
break;
}
}
if ( (signed int)v6 >= dest_safestring.length )
LODWORD(v6) = dest_safestring.length - 1;
v9 = (signed int)v6;
memcpy_0(dest_str, source_str, (signed int)v6);
dest_str[v9] = sead::SafeStringBase<char>::cNullChar;
}
```
⬇️
```cpp
dest_safestring = source_safestring;
```
Note: the while loop comes from an inlined version of sead::SafeString::calcLength.
---
##### sead::SafeString::startsWith
```cpp
if ( sead::SafeStringBase<char>::cNullChar != 'E' )
{
v11 = string->cstr;
v12 = "nemy";
v13 = 'E';
while ( (unsigned __int8)*v11 == v13 )
{
v14 = (unsigned __int8)*v12++;
v13 = v14;
++v11;
if ( v14 == (unsigned __int8)sead::SafeStringBase<char>::cNullChar )
goto LABEL;
}
foo();
}
LABEL:
bar();
```
⬇️
```cpp
if (string.startsWith("Enemy"))
bar();
else
foo();
```
This weird optimization can lead to malformed strings in Hex-Rays's output for strings that contain Japanese or multibyte characters more generally.
---
##### sead::SafeString::isEqual / operator==
```cpp
enemy->vptr->assureTermination(enemy);
enemy->vptr->assureTermination(enemy);
v15 = enemy->cstr;
if ( v15 == "Enemy_Assassin_Junior" )
{
LABEL_37:
foo();
}
else
{
v16 = 0LL;
do
{
v17 = (unsigned __int8)v15[v16];
if ( v17 != (unsigned __int8)aEnemyAssassinJ[v16] )
break;
if ( v17 == (unsigned __int8)sead::SafeStringBase<char>::cNullChar )
goto LABEL_37;
++v16;
}
while ( v16 < 0x80001 );
bar();
}
```
⬇️
```cpp
// enemy is a sead::SafeString or a derived class
if (enemy == "Enemy_Assassin_Junior")
foo();
else
bar();
```
---
##### sead::SafeString::isEmpty
```cpp
if ( *string.cstr == sead::SafeStringBase<char>::cNullChar )
```
⬇️
```cpp
if (string.isEmpty())
```
---
#### ScopedLock
```cpp
sead::CriticalSection::lock(foo);
bar();
sead::CriticalSection::unlock(foo);
```
⬇️
```cpp
{
auto lock = sead::makeScopedLock(foo);
bar();
}
```
---
#### Buffer
An sead::Buffer is a non-owning view over a contiguous array of values: it is essentially a wrapper around a raw pointer and a size count.
##### sead::Buffer::allocBufferAssert
In this example, 0x108 is the size of each item.
```cpp
v9 = 0x108LL * (signed int)num_tables;
v10 = is_mul_ok((signed int)num_tables, 0x108uLL) == 0;
v11 = __CFADD__(v9, 8LL);
v12 = v9 + 8;
if ( v11 )
v13 = 1;
else
v13 = 0;
if ( (unsigned int)v10 | v13 )
size = 1LL;
else
size = v12;
v15 = (char *)operator new[](size, &heap->_, 8u, &std::nothrow);
if ( v15 )
{
*(_QWORD *)v15 = (signed int)num_tables;
v16 = (BdropTable *)((char*)v15 + 8);
// loop over each item and call a constructor
// note: the constructor may be inlined
// at the end:
buffer->size = num_tables;
buffer->data = v16;
}
```
⬇️
```cpp
buffer->allocBufferAssert(num_tables, heap);
```
Another code pattern with a multiplication that looks different:
```cpp
v13 = operator new[](0x28LL * (unsigned int)count + 8, &heap->_, 8u, &std::nothrow);
if ( v13 )
{
*v13 = count;
v14 = (signed __int64)(v13 + 1);
// loop over each item and call a constructor
// note: the constructor may be inlined
// at the end:
*buffer = count;
*((_QWORD *)buffer + 1) = v14;
}
```
⬇️
```cpp
buffer->allocBufferAssert(count, heap);
```
If each item is a trivially constructible type (e.g. the buffer stores ints or pointers) then there will be no loop that calls a constructor and the compiler will not store the size of the array in the first 8 bytes of the allocation.
##### sead::Buffer::operator[]
With automatic bounds checks.
```cpp
if ( buffer->size <= i )
item = buffer->data;
else
item = buffer->data[i];
```
⬇️
```cpp
item = buffer[i];
```
---
#### RTTI (Runtime Type Info)
##### sead::DynamicCast
```cpp
some_ptr = ...;
x = __ldar(...); // usually a guard variable, but the variable is not always named
another_ptr = some_ptr;
if ( (x & 1) == 0 && _cxa_guard_acquire_0(...)) // the same guard variable
{
... = &...;
_cxa_guard_release_0(...); // the same guard variable
}
if ( another_ptr && another_ptr->checkDerivedRuntimeTypeInfo(another_ptr, ...) )
{
// code that uses another_ptr
}
```
⬇️
```cpp
if (auto* another_ptr = sead::DynamicCast<T>(some_ptr))
...
```
T is a derived type that should be related to the type of the original pointer.
### agl
agl is one of Nintendo's in-house graphics libraries.
#### Parameter utilities
In *Breath of the Wild*, its parameter utilities are heavily used for the game's configuration files.
##### agl::utl::Parameter::init
```cpp
name.vptr = &sead::SafeString::vt;
name.cstr = "Item";
label.vptr = &sead::SafeString::vt;
label.cstr = "表示距離";
meta.vptr = &sead::SafeString::vt;
meta.cstr = (char *)&nullbyte;
agl::utl::ParameterBase::initializeListNode(
&foo._,
&name,
&label,
&meta,
bar);
foo.value = default_value;
```
⬇️
```cpp
foo.init(default_value, "Item", "表示距離", bar);
```
---
##### agl::utl::IParameterObj::applyResParameterObj
```cpp
agl::utl::IParameterObj::applyResParameterObj_(foo, 0, bar, 0LL, 0.0, 0LL);
```
⬇️
```cpp
foo.applyResParameterObj(bar);
```
---
##### agl::utl::ResParameterArchive::getRootList
```cpp
agl::utl::ResParameterArchive::ResParameterArchive(&archive, data);
root.ptr = (agl::utl::ResParameterListData *)((char *)&archive.ptr[1] + (unsigned int)archive.ptr->rootOffsetAfterHeader);
```
⬇️
```cpp
agl::utl::ResParameterArchive archive{data};
auto root = archive.getRootList();
```
---
##### Getting ResParameterObj or ResParameterList
```cpp
key.vptr = &sead::SafeString::vt;
key.cstr = "InvalidWeathers";
key_hash = agl::utl::ParameterBase::calcHash(&key);
idx = agl::utl::ResParameterList::searchObjIndex(&foo, key_hash);
if ( idx == 0xFFFFFFFF )
{
obj = 0LL;
}
else
{
obj = (__int64)v30.ptr + 8 * idx + 4 * (unsigned __int16)v30.ptr->objOffsetNum;
if ( obj )
{
bar();
}
}
```
⬇️
```cpp
const auto obj = agl::utl::getResParameterObj(foo, "InvalidWeathers");
if (obj.ptr())
bar();
```
### Havok
### operator delete overload
```c++
void foo(void* this) {
if ( this )
{
TlsValue = nn::os::GetTlsValue(hkMemoryRouter::s_memoryRouter);
(*(void (__fastcall **)(_QWORD, __int64, __int64))(**(_QWORD **)(TlsValue + 0x58) + 0x18LL))(
*(_QWORD *)(TlsValue + 0x58),
this,
0x40LL);
}
}
```
or after setting pointer types correctly:
```c++
void foo(void* this)
{
if ( this )
{
router = (hkMemoryRouter *)nn::os::GetTlsValue(hkMemoryRouter::s_memoryRouter);
router->m_heap->blockFree(router->m_heap, (void *)this, 0x40);
}
}
```
This is just a standard Havok class deleting/D0 destructor. You can get it with the HK_DECLARE_CLASS_ALLOCATOR macro.
### hkVector4f
#### hkVector4f::setCross (cross product)
Reminder: the cross product is antisymmetric.
```c++
v9 = vextq_s8((int8x16_t)lhs, (int8x16_t)lhs, 4uLL);
v10 = vextq_s8((int8x16_t)rhs, (int8x16_t)rhs, 4uLL);
v9.n128_u64[1] = vrev64_s32((int32x2_t)vextq_s8(v9, v9, 8uLL).n128_u64[0]).n64_u64[0];
v10.n128_u64[1] = vrev64_s32((int32x2_t)vextq_s8(v10, v10, 8uLL).n128_u64[0]).n64_u64[0];
cross = vsubq_f32(vmulq_f32(lhs, (float32x4_t)v10), vmulq_f32(rhs, (float32x4_t)v9));
result = (float32x4_t)vextq_s8(vextq_s8((int8x16_t)cross, (int8x16_t)cross, 0xCuLL), (int8x16_t)cross, 8uLL);
```
⬇️
```c++
hkVector4f result;
result.setCross(lhs, rhs);
```
#### hkVector4f::dot<3> (dot product)
```c++
dot = (int8x16_t)vmulq_f32(lhs, rhs);
dot.n128_u64[0] = vpadd_f32((float32x2_t)dot.n128_u64[0], (float32x2_t)vextq_s8(dot, dot, 8uLL).n128_u32[0]).n64_u64[0];
dot.n128_f32[0] = vpadd_f32((float32x2_t)dot.n128_u64[0], (float32x2_t)dot.n128_u64[0]).n64_f32[0];
```
⬇️
```c++
hkSimdReal dot = lhs.dot<3>(rhs);
```
#### hkVector4f::lengthSquared<3>
This is a special case of hkVector4f::dot<3>.
```c++
len = vmulq_f32(vec, vec);
len.n128_u64[0] = vpadd_f32((float32x2_t)len.n128_u64[0], (float32x2_t)vextq_s8(len, len, 8uLL).n128_u32[0]).n64_u64[0];
len.n128_u64[0] = vpadd_f32((float32x2_t)len.n128_u64[0], (float32x2_t)len.n128_u64[0]).n64_u64[0];
```
⬇️
```c++
hkSimdReal len = vec.lengthSquared<3>();
```
#### hkVector4f::lengthInverse<3>
This is hkVector4f::lengthSquared<3> followed by a square root inverse calculation.
```c++
// start of lengthSquared
len = vmulq_f32(vec, vec);
len.n128_u64[0] = vpadd_f32((float32x2_t)len.n128_u64[0], (float32x2_t)vextq_s8(len, len, 8uLL).n128_u32[0]).n64_u64[0];
len.n128_u64[0] = vpadd_f32((float32x2_t)len.n128_u64[0], (float32x2_t)len.n128_u64[0]).n64_u64[0];
// end of lengthSquared
// start of hkSimdFloat32::sqrtInverse
v10.n128_u64[0] = vrsqrte_f32((float32x2_t)len.n128_u64[0]).n64_u64[0];
v10.n128_u64[0] = vmul_f32(
(float32x2_t)v10.n128_u64[0],
vrsqrts_f32(
(float32x2_t)norm.n128_u64[0],
vmul_f32((float32x2_t)v10.n128_u64[0], (float32x2_t)v10.n128_u64[0]))).n64_u64[0];
inverse.n128_u64[0] = vbic_s8(
vmul_f32(
vrsqrts_f32(
(float32x2_t)norm.n128_u64[0],
vmul_f32((float32x2_t)v10.n128_u64[0], (float32x2_t)v10.n128_u64[0])),
(float32x2_t)v10.n128_u64[0]),
vclez_f32((float32x2_t)norm.n128_u64[0])).n64_u64[0];
inverse.n128_u64[1] = inverse.n128_u64[0];
// end of hkSimdFloat32::sqrtInverse
```
⬇️
```c++
hkSimdReal inverse = vec.lengthInverse<3>();
```
#### hkVector4f::normalize<3>
This is lengthInverse<3> followed by a multiplication.
```c++
// start of lengthSquared
len = vmulq_f32(vec, vec);
len.n128_u64[0] = vpadd_f32((float32x2_t)len.n128_u64[0], (float32x2_t)vextq_s8(len, len, 8uLL).n128_u32[0]).n64_u64[0];
len.n128_u64[0] = vpadd_f32((float32x2_t)len.n128_u64[0], (float32x2_t)len.n128_u64[0]).n64_u64[0];
// end of lengthSquared
// start of hkSimdFloat32::sqrtInverse
v10.n128_u64[0] = vrsqrte_f32((float32x2_t)len.n128_u64[0]).n64_u64[0];
v10.n128_u64[0] = vmul_f32(
(float32x2_t)v10.n128_u64[0],
vrsqrts_f32(
(float32x2_t)norm.n128_u64[0],
vmul_f32((float32x2_t)v10.n128_u64[0], (float32x2_t)v10.n128_u64[0]))).n64_u64[0];
inverse.n128_u64[0] = vbic_s8(
vmul_f32(
vrsqrts_f32(
(float32x2_t)norm.n128_u64[0],
vmul_f32((float32x2_t)v10.n128_u64[0], (float32x2_t)v10.n128_u64[0])),
(float32x2_t)v10.n128_u64[0]),
vclez_f32((float32x2_t)norm.n128_u64[0])).n64_u64[0];
inverse.n128_u64[1] = inverse.n128_u64[0];
// end of hkSimdFloat32::sqrtInverse
vec = vmulq_f32(inverse, vec);
```
⬇️
```c++
vec.normalize<3>();
```
## C++ features
### Classes
#### Constructors
If you see a function that modifies the vtable pointer and/or calls a lot of other constructors, chances are that you are dealing with a constructor.
In C++, most of the code in constructor functions tends to be automatically generated by the compiler. For example, the following function:
```cpp
void __fastcall ksys::res::Handle::Handle(ksys::res::Handle *this)
{
this->mFlags = 1;
this->mStatus = 0;
this->mUnit = 0LL;
this->vtable = &ksys::res::Handle::vt;
ksys::util::ManagedTaskHandle::ManagedTaskHandle(&this->mTaskHandle);
this->field_40 = 0LL;
this->field_48 = 0LL;
}
```
is automatically generated based on the class definition:
```cpp
// irrelevant details were simplified or removed
struct Handle {
u8 mFlags = 1;
Status mStatus = Status::_0;
ResourceUnit* mUnit = nullptr;
util::ManagedTaskHandle mTaskHandle;
sead::ListNode mListNode;
};
```
Note that the sead::ListNode constructor was inlined here, and *that* constructor was also automatically generated by the compiler:
```cpp
class ListNode
{
public:
// ...
ListNode* mPrev = nullptr;
ListNode* mNext = nullptr;
};
```
---
#### Member functions and member variables
C++ member functions are called as if they had the `this` pointer as the first argument.
```cpp
sead::CriticalSection::lock(some_variable);
ksys::res::ResourceUnit::attachHandle(unit, handle);
```
⬇️
```cpp
some_variable->lock();
unit->attachHandle(handle);
```
Member variables are accessed using the `this` pointer. In C++, explicitly writing `this` is usually unnecessary:
```cpp
if ( this->mNumCaches <= idx )
cache = this->mCaches;
else
cache = &this->mCaches[idx];
```
⬇️
```cpp
if (mNumCaches <= idx)
cache = mCaches;
else
cache = &mCaches[idx];
```
---
### Custom operator new
sead defines several custom allocation functions. You should `#include <basis/seadNew.h>` to ensure they are used for heap allocations.
```cpp
void* ptr = ::operator new(SOME_SIZE_HERE, heap, 8u);
SomeClass::SomeClass(ptr); //< constructor call
```
⬇️
```cpp
auto* ptr = new (heap) SomeClass;
```
Sometimes, a non-throwing overload of `operator new` is used:
```cpp
void* ptr = ::operator new(SOME_SIZE_HERE, heap, 8u, &std::nothrow);
SomeClass::SomeClass(ptr); //< constructor call
```
⬇️
```cpp
auto* ptr = new (heap, std::nothrow) SomeClass;
```
This also applies for `new[]`.

314
Contributing.md
View File

@ -1,315 +1,3 @@
# Contributing
To contribute to the project, you will need a disassembler or a decompiler such as Hex-Rays or Ghidra.
Experience with reverse engineering optimized C++ code is very useful but not necessary if you already know how to decompile C code.
Feel free to join the [Zelda Decompilation](https://discord.zelda64.dev/) Discord server if you have any questions.
## Language
This project is written in C++17. (Technically, in a subset of C++17, since Clang 4.0.1 supports most of C++17 but not all of it.)
Unlike the vast majority of games that are being decompiled in 2021, *Breath of the Wild* is a large modern game that is written in C++. While C and C++ have a similar syntax, C++ is somewhat more complex than C. To avoid getting lost in C++ code, please familiarize yourself with the following basic language concepts *before* decompiling:
* [namespaces](https://en.cppreference.com/w/cpp/language/namespace)
* Instead of using prefixes such as `z_` to avoid name conflicts, C++ code instead relies on namespaces.
* [classes](https://en.cppreference.com/w/cpp/language/class), including inheritance, polymorphism, virtual functions
* C++ classes/structs are basically C structs on steroids. Notably, C++ classes can contain member functions.
* Member functions get an implicit `this` argument, which is passed as if it were the first argument.
* [Virtual member functions](https://en.cppreference.com/w/cpp/language/virtual) are member functions that can be overridden in derived classes.
* Virtual member functions are usually implemented with a virtual function table or "vtable", which is a table of function pointers.
* [const correctness](https://isocpp.org/wiki/faq/const-correctness) for member functions
* iterators and range-based for loops (e.g. `for (int i : my_list_of_ints) {}`)
## Editor/IDE setup
BotW is mostly set up like a normal C++ project using standard build tools and compilers like Clang, CMake, Ninja, etc. so autocomplete and "IntelliSense" style features should work almost out-of-the-box.
### VSCode
Make sure you have the C++ and the CMake Tools extensions installed and enabled. And then just answer "yes" when you're asked whether you would like CMake Tools to configure IntelliSense for you.
In certain circumstances, VSCode may silently attempt to set the build type to Debug. Make sure the build type is set to RelWithDebInfo (and not Debug); otherwise the entire build will mismatch.
### CLion
CLion interacts with CMake directly, so you need to make sure CLion's build profile is configured correctly.
1. Open the Settings window and go to the Build > CMake pane.
2. Remove all existing build profiles, and add a new build profile (call it whatever you want):
* Build type: RelWithDebInfo
* CMake options: `-DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_TOOLCHAIN_FILE=toolchain/ToolchainNX64.cmake -GNinja`
* Build directory: `build`
3. Press OK; CLion will automatically reload the CMake project.
## Decompiler setup
Using a decompiler (a tool that translates assembly into pseudocode) such as Hex-Rays or Ghidra is strongly recommended.
As of 2022, IDA/Hex-Rays is (somewhat subjectively) still ahead of other tools for heavy analysis of large C++ AArch64 binaries,
so we recommend acquiring a copy of IDA Pro 7.6+ if you want to contribute to BotW.
### IDA (recommended)
#### Useful plugins
- [LazyIDA](https://gist.github.com/leoetlino/bdac084a1fb0342b734faecf3ae49df9) to copy function addresses with a single key press (W)
- [HexRaysPyTools](https://github.com/leoetlino/HexRaysPyTools) for automatic reconstruction of structs and other useful features in the decompiler view
#### Loading the database
If you have IDA 7.6+, ping @leoetlino on the Zelda Decompilation Discord to get a copy of the IDA database (IDB) which will make decompilation easier and help with understanding the code.
#### Usage
When you open the IDB, you'll see several tabs:
* IDA View: This is an interactive disassembly of the executable.
* Pseudocode: This is where you can find the pseudocode that Hex-Rays produces. Unlike other decompilers such as m2c, this output is fully interactive. You can define function signatures, variable names, types, etc. in this tab and improve the pseudocode output interactively. This is the tab you'll be working in most of the time. (If you can't find this tab or if you accidentally closed it, go to the IDA View tab, select a function and then press F5 to re-open the pseudocode tab.)
* Strings: A list of all valid strings in the executable. Occasionally useful for finding functions that have not been reverse engineered and named yet.
* Structures: A listing of all defined structures in the IDB. Useful for defining structures/fields and making the pseudocode easier to read.
Common keyboard shortcuts:
* Ctrl+P brings up a function chooser. Very useful for the IDA View and Pseudocode tabs.
* Use Ctrl+Shift+Up/Down to show the previous/next function.
* Ctrl+W saves the database.
* To rename an item, click on its text and press N.
* To change the type of a variable (in the pseudocode) or a function, click on its text and press Y.
* Press H on a number to switch between its decimal and hexadecimal representations.
* [Other shortcuts are mentioned here](https://www.hex-rays.com/products/ida/support/freefiles/IDA_Pro_Shortcuts.pdf).
* [If you have HexRaysPyTools] Shift+L to propagate types from a call site (invocation) to the callee function.
* [If you have LazyIDA] W to copy the address of the current selection (put the cursor on the first line of the pseudocode to copy the address of the current function)
### Ghidra
[Ghidra](https://ghidra-sre.org/) is an open-source software reverse engineering tool developed by the NSA.
If you cannot or do not want to use IDA, Ghidra is a decent alternative (though less ideal than IDA for RE'ing something like BotW).
Note that you will need to import names and types manually and you will not be able to make use of the existing IDA reverse engineering database if you use Ghidra, so this is really not the recommended option.
#### Loading the executable
1. Install the [Switch loader](https://github.com/Adubbz/Ghidra-Switch-Loader).
2. Open the 1.5.0 NSO. If you've run the setup script (as mentioned in the README), a copy of the NSO is stored at `data/main.nso`.
3. Wait for Ghidra to analyse the entire executable. This can take a long time.
4. Use the script in `tools/common/ghidra_scripts` to import function names from this project.
## How to decompile
0. Build the project and run `tools/check` to make sure the project is set up correctly. You should see "OK" at the end.
1. **Pick a function that you want to decompile.**
* Prefer choosing a function that you understand or that is already named in your IDA/Ghidra database.
* Use our [Trello project board](https://botw.link/trello) to figure out what needs to be decompiled. Make sure it's not already being worked on by somebody else!
* The "Blocked" label means that the task cannot be easily done at the moment because it requires something else to be decompiled or stubbed first.
* "Easy" tasks are recommended to familiarize yourself with the process. They can typically be done pretty quickly.
* "Requires library integration" tasks require decompiling an external library (e.g. agl, sead, ...) and integrating it into the project.
* "Manager/singleton" means that the task is about a manager or a singleton (a class with only a single instance).
* If you want to work on libraries rather than on BotW code, take a look at [this board](https://botw.link/trello-libs)!
* Search for the card's label in IDA to locate relevant functions to decomp. If you can't find any good match, feel free to ask somebody to clarify the task on the help channel on Discord.
* You do not need to fully understand the function, but you should at least have a rough idea of what it does.
* If you are feeling more ambitious, pick an entire C++ class! This usually allows understanding the code better.
2. **Try to understand** what the function does using Hex-Rays or Ghidra.
* Understanding the function is very important.
* Rename variables, add structures, do everything you can to make the output as clean as possible.
* C++ code tends to make heavy use of inline functions. For example, inlined string comparisons or copies are very common and tend to obscure what the function does. Focus on the outline of the function.
* The [cheatsheet](Cheatsheet.md) can help you recognize inline functions.
3. **Implement the function in C++.**
* Stay close to the original code, but not too close: your code should mostly look like normal, clean C++ code. If it does not, chances are that you won't get a good match at all.
* Do **NOT** copy and paste any pseudocode. **Reimplement it**. While we cannot go for a fully "clean room" approach, you should be reimplementing code, not copy/pasting anything from the original executable.
* PRs that violate this rule will be rejected.
* You usually have a lot of leeway when reimplementing a function, but some things must be kept the same in your reimplemented version in order to have any chance of getting a matching function:
* _Function calls_. You should not add or remove non-inlined function calls.
* Struct/class member variable offsets.
* Be careful with float comparisons. Because of float semantics, `if (x < y) f(); else g();` and `if (x >= y) g(); else f();` are **not** functionally equivalent (because they behave differently if one of the floats is NaN).
* Things that you can change in your reimplemented version:
* _Names_. Some of the function/variable names are just placeholder, so feel free to use your own names if you think they are better.
* _Functions_. You are free to split a function into several smaller functions or introduce utility functions, even if there isn't an explicit function call in the original code, as long as your reimplemented functions are getting inlined.
* Note that LLVM will usually not inline functions if they are too large.
* `if (x) f(); else g();` and `if (!x) g(); else f();` generally produce the same code. Use [early exits](https://llvm.org/docs/CodingStandards.html#use-early-exits-and-continue-to-simplify-code) when possible.
* Keep in mind that decompilers can only produce C pseudocode. Some function calls may be C++ member function calls.
* Identify inlined functions and *uninline* them. For example, if you see a string copy, do **not** write the copy loop manually! Instead, call the inline function and let the compiler inline the function for you.
* Identify duplicate pieces of code: those are usually a sign that functions have been inlined.
* Non-inline function calls can just be stubbed if you don't feel like decompiling them at the moment. To "stub" a function, just declare the function (and the enclosing class/namespace/etc. if needed) without implementing/defining it.
* Follow the [coding style guidelines](#code-style) where applicable.
4. **Build**.
5. **Add the function name to the list of decompiled functions.**
* To do so, open `data/uking_functions.csv`, search for the name or the address of function you have decompiled, and add the function name to the last column.
* Example: `0x00000071010c0d60,U,136,BaseProcMgr::createInstance`
6. **Compare the assembly** with `tools/check -mw <function name>`
* This will bring up a two-column diff. The code on the left is the original code; the code on the right is your version of the function.
* You may ignore address differences (which often show up in adrp+ldr pairs or bl or b).
* If you modify a source file while the diff is visible, it will be automatically rebuilt and the diff will update to match the new assembly code.
* Remove `-mw` from the command if you do not want automatic rebuilds.
* Other useful flags:
* To show C++ source code interleaved with the assembly in the diff, pass `-c` or `--source`.
* To get a three-column diff (original, decomp, diff with last decomp attempt), pass `-3` (do not use with `-c`).
7. **Tweak the code to get a perfectly matching function**.
* Clang is usually quite reasonable so it is very common for functions -- even complicated code -- to match on the first try.
* **Focus on large differences.** If you have large differences (e.g. entire sections of code being at the wrong location), focus on getting rid of them first and ignore small differences like regalloc or trivial reorderings.
* **Regalloc:** If you only have regalloc differences left in a function that *looks* semantically equivalent, double-check whether it is truly equivalent: such differences are typically caused by using the wrong variable. It is rare for LLVM to use a different set of registers if the code is equivalent.
* This is usually the most difficult part of matching decomp. Please ask on Discord if you need help!
* The [cheatsheet](Cheatsheet.md) might help you recognize code patterns and contains a checklist for common matching issues.
8. **Update the list of decompiled functions**.
* If you have a function that matches perfectly, great!
* If there are still minor differences left, write a comment to explain what is wrong (if you think that is necessary), and change the status (the second column) to `m` (minor difference) in the CSV.
* For major differences (lots of entirely red/green/blue lines in the diff), use a capital `M` (major difference) in place of `m`.
9. Before opening a PR, reformat the code with clang-format and run `tools/check`.
* You can use clang-format via your editor – VSCode and CLion have built-in clang-format support — or by calling `git clang-format` (for files you have `git add`ed and not yet committed).
* If your editor does not have built-in support for clang-format, or if you need to invoke clang-format in a terminal, you'll need to install it manually.
* If your Linux distro or system (e.g. macOS) does not package clang-format 12, you can download it from [the LLVM project website here](https://releases.llvm.org/download.html)
## Importing names or types from source code
In IDA, run the `tools/common/rename_functions_in_ida.py` script (File > Script file...) to import names from the decomp source code back into the IDA database. This can be done as often as you want.
To import types into the IDB, you can use [classgen](https://github.com/leoetlino/classgen) ([binary builds available here](https://github.com/leoetlino/classgen/releases/latest)) or IDA Pro 7.7's Clang-based header parser (less convenient).
## Code style
BotW has 40MB of code and contributors *need* to read and modify existing parts of the codebase very often: inconsistencies lead to a loss of efficiency, and we literally cannot afford that considering our small number of contributors. To avoid wasting time on formatting issues, we use clang-format to automatically enforce a consistent coding style.
Before opening a PR, please format your code with clang-format 12 and ensure the following guidelines are followed. This will allow your contributions to be reviewed more quickly.
### General
* Lines should not be longer than 100 characters.
* Use 4 spaces to indent.
* Use `nullptr`, not `NULL` or `0`.
* Only use `auto` if the variable type is obvious, too long to type or if it doesn't matter.
* To compare an integer against zero, write `if (value == 0)`, not `if (!value)`. (This rule doesn't apply to booleans.)
* To compare a value against nullptr, either `if (pointer != nullptr)` or `if (pointer)` is fine.
### Header files
* Use `#pragma once` for header guards.
* Avoid unnecessary includes. Forward declare types when possible to reduce compilation times.
### Includes
* Use `#include "..."` when including U-King (BotW) header files. KingSystem (ksys) is treated as being part of BotW.
* Include paths should be relative to src/.
* OK: `#include "KingSystem/ActorSystem/actActor.h"`
* Not OK: `#include "actActor.h"`
* Use `#include <...>` for system or library includes. Examples:
* Standard C++ library headers (e.g. `<optional>`, `<type_traits>`, `<limits>`, ...)
* sead (e.g. `<prim/seadSafeString.h>`)
* Other Nintendo libraries like agl, evfl, eui, ...
### Naming
* Type names (classes/structs, enums, typedefs/alias declarations) and compile-time constants should be in UpperCamelCase.
* `class ActorInfoData`
* `using Manager = ksys::gdt::Manager;`
* `constexpr int NumActors = 42;`
* Function names should be in camelCase.
* `void doStuff()`
* `void SomeClass::doThis()` (for a member function)
* Variables should be in lowercase_snake_case, except for class member variables, which should be prefixed with 'm' and writtenLikeThis.
* `int a_dummy_variable = 42;`
* `void test(int first_argument, bool second_argument) { ... }`
* `class SomeClass { int mMemberVariable; };` (m prefix + camelCase)
* `struct SomeStruct { int member_variable; };` (regular snake_case)
* Static variables should be prefixed with 's' and globals with 'g'.
* `s_static_variable`
* `sStaticVariable` if it's a static member variable
### Classes
* Ordering
* Put access specifiers in this order: `public`, `protected`, `private`.
* Declare member functions in this order: constructor, destructor, operators, other member functions.
* Declare non-static member variables after function declarations.
* Declare static member variables before non-static variables.
* Virtual functions need to match the original order in the executable, though, so ignore this rule if following it would require reordering virtual functions.
* If a class uses a macro like `SEAD_SINGLETON_DISPOSER` or one of the SEAD_RTTI macros, put the macro right after the opening brace, before `public:`.
* Use `= default;` instead of constructors/destructors with an empty body.
* Use the `override` keyword instead of `virtual` when overriding virtual functions from a parent class.
* Mark member functions as const if they do not modify any non-static member variables.
* Do not use `this->` to refer to member variables or member functions unless it is necessary. (It is usually unnecessary.)
```cpp
class Test {
SEAD_RTTI_BASE(Test)
public:
Test();
virtual ~Test() = default;
virtual bool isTest() const { return true; }
void doStuff() {}
private:
static bool sFoo = false;
bool mMemberVariable = true;
};
class TestDerived : public Test {
public:
bool isTest() const override { return false; }
};
```
## Non-inlined functions
When **implementing non-inlined functions**, please compare the assembly output against the original function and make it match the original code. At this scale, that is pretty much the only reliable way to ensure accuracy and functional equivalency.
However, given the large number of functions, certain kinds of small differences can be ignored when a function would otherwise be equivalent:
* Regalloc differences.
* Warning: ensure that the return type of the function is correct. Differences that involve the X0-X7, W0-W7 or S0-S3 registers at the end of a function are suspicious.
* Instruction reorderings when it is obvious the function is still semantically equivalent (e.g. two add/mov instructions that operate on entirely different registers being reordered)
## Header utilities or inlined functions
For **header-only utilities** (like container classes), use pilot/debug builds, assertion messages and common sense to try to undo function inlining. For example, if you see the same assertion appear in many functions and the file name is a header file, or if you see identical snippets of code in many different places, chances are that you are dealing with an inlined function. In that case, you should refactor the inlined code into its own function.
Also note that introducing inlined functions is sometimes necessary to get the desired codegen.
If a function is inlined, you should try as hard as possible to make it match perfectly. For inlined functions, it is better to use weird code or small hacks to force a match as differences would otherwise appear in every single function that inlines the non-matching code, which drastically complicates matching other functions. If a hack is used, wrap it inside a `#ifdef MATCHING_HACK_{PLATFORM}` (see below for a list of defines).
### Matching hacks
This project sometimes uses small hacks to force particular code to be generated by the compiler. Those have no semantic effects but can help with matching assembly code especially when the hacks are used for functions that are inlined.
* `MATCHING_HACK_NX_CLANG`: Hacks for Switch, when compiling with Clang.
## Modifying library code
Changes to the following libraries must be PR'd/submitted to their own repository:
* sead: https://github.com/open-ead/sead
* NintendoSDK: https://github.com/open-ead/nnheaders
* agl: https://github.com/open-ead/agl
* EventFlow: https://github.com/open-ead/EventFlow
## Project tools
* Check all decompiled functions for issues: `tools/check`
* To compare assembly: `tools/check <mangled function name>`
* The function **must be listed in data/uking_functions.csv first**.
* To do so, search for the name or the address of function you have decompiled, and add the mangled function name to the last column.
* Pass the `--source` flag to show source code interleaved with assembly code.
* Add the `--inlines` flag to show inline function calls. This is not enabled by default because it usually produces too much output to be useful.
* Pass `-mw3` for automatic rebuilds whenever a source file is modified.
* For more options, see [asm-differ](https://github.com/simonlindholm/asm-differ).
* To print progress: `tools/common/progress.py`
* Note that progress is only approximate because of inline functions, templating and compiler-generated functions.
* To print AI class decompilation status: `tools/ai_progress.py`
* Use this to figure out which AI classes have not been decompiled yet.
* To list symbols: `tools/listsym` (pass --help to see available options)
[Contributing guides are available on our website](https://botw.link/contribute).

214
README.md
View File

@ -20,216 +20,4 @@ This is an experimental, WIP decompilation of *The Legend of Zelda: Breath of th
The goal of this project is to better understand game internals, aid with glitch hunting and document existing knowledge in a permanent, unambiguous form which helps further reverse engineer the game.
Considering the large size of the executable (~40MB), it is not expected to reach 100% progress within a reasonable timeframe.
As a result, the project is unlikely to produce a working executable in the near future. It will help with understanding and reverse engineering the game even in its incomplete state, but it will **not** help with playing BotW or porting the game to other platforms, which is **explicitly a non-goal**.
Progress: https://botw.link/progress
## Scope
This project only concerns the main executable which contains all the game code and statically linked libraries. The RomFS and the SDK libraries are *out of the scope* of this project.
* Main executable (main NSO)
* *Breath of the Wild* code
* Actual game code (`Game` / uking:: namespace)
* Framework/engine code (`KingSystem` / ksys:: namespace)
* Statically linked libraries
* First-party libraries (e.g. sead, agl, EventFlow, etc.)
* NintendoSDK inlined utilities
* Any other statically linked library, except:
* libcurl
* NintendoSDK-NEX
* Havok (physics engine)
Excluded libraries will not be fully decompiled but may be partly re-implemented or decompiled, and (reverse-engineered) headers will still be provided so that the rest of the codebase can still use those libraries.
## Frequently Asked Questions
### What is (matching) decompilation?
Decompiling is the process of turning the game's compiled code back into equivalent, readable C++ source code.
Matching decompilation goes one step further and produces original source code that **compiles to the exact same assembly**. This process makes functional correctness extremely easy to verify since the assembly can simply be compared against the original executable. Thanks to Clang, this is easier than one would expect.
#### Matching decomp in this project
Given the impossibility of automatically splitting the assembly and generating a matching binary (as is done in many other decomp projects), the sheer size of the main executable and the usage of many software libraries, this project takes a somewhat experimental approach to matching decompilation.
Because meaningfully splitting the code is not feasible, the built executable currently only contains functions that have been decompiled and no effort is being made to put functions and data at the correct addresses.
Instead of trying to match the entire executable, each function is matched individually and source code is organized in whichever way makes the most sense. Libraries are not treated as being part of the game code, but as external dependencies. The result is that the codebase looks a lot more like a regular software project than a decompilation codebase. Since C++ code makes heavy use of inline functions and zero-cost abstractions that disappear in compiled code, contributors have a lot more leeway when it comes to organizing files and adding abstractions.
### How do you name things?
File names, class or function names, and the file organization come from leftover strings. Unlike some other first-party games such as *Super Mario Odyssey*, all known public versions of U-King are completely stripped, so most names are just more or less educated guesses that try to fill in the blanks.
As more parts of the game get decompiled, it becomes easier to figure out what the rest of the game is doing and equally easier to name functions. It's a positive feedback loop.
### How easy is it to match functions?
Compared to other decomp projects for older compilers: **extremely easy**. Clang is an extremely reasonable compiler with much fewer memes than older compilers such as IDO or older versions of GCC:
* Stack reordering issues are extremely rare, given that AArch64 uses its registers a lot more efficiently. Even when the stack is used, things Just Work™ in the vast majority of cases.
* Pure register allocation (regalloc) issues are almost non-existent. If you see something that looks like a regalloc problem, it usually means your code is not semantically equivalent.
* No `if (1)` shenanigans.
* No "same line" memes (codegen being different if two statements are put on the same line).
* Whitespace doesn't matter.
In general, two equivalent constructs that *should* clearly produce the same code actually produce the exact same code. There are exceptions, of course, but many things simply do not matter at all for matching. Inline functions do often affect codegen, though.
Getting perfect matches on the first try happens pretty routinely, even for medium-sized and large functions (>1kB).
Most functions tend to call several other inline functions, notably utility functions from sead; as many core sead modules have already been reversed, decompiling a function sometimes only requires recognizing those function calls: decompilation at a higher level of abstraction!
### Doesn't this mean the end result will be quite inaccurate?
Even though Clang is generally less picky than ancient compilers, it is still able to reveal a lot of interesting information about how the original source code was written.
Granted, Clang will *not* reveal things such as whether developers put two statements on the same line, but in many cases we can still figure how they organised the source code and whether separate inline functions or classes were used.
Coupled with all the strings and all the information C++ language features leak (e.g. classes, virtual functions, hierarchy, etc.), we actually still get relevant, useful info (e.g. abstractions) that *actually* helps understand a modern codebase.
It is also important to remember that this is still a matching decompilation, which means that the code already perfectly matches the original *assembly*; matching functions are guaranteed to be fully equivalent to the original code, and thanks to Clang being picky *where it matters* most of the abstractions should match the original *source*.
### What version is being decompiled?
The Switch 1.5.0 version. Working with Clang is so much nicer than working with the Wii U's proprietary compiler (GHS). And at least an order of magnitude nicer than dealing with most compilers that are used for other matching decomp projects for that matter.
Having access to the Wii U 1.5.0 version is occasionally useful though, because GHS often optimizes and inlines differently, which can help reveal the existence of inline functions.
#### Why not decompile 1.6.0?
Because of aggressive compiler optimisations and severe code bloat, 1.6.0 would be extremely painful to reverse engineer, let alone decompile. [See here](https://gist.github.com/leoetlino/3156c286c7232c2e73b9b74b91021353) for a comparison between 1.5.0 and 1.6.0 code of the main function (called `nnMain`).
(The culprit is link time optimisation, which allows LLVM to perform extremely aggressive inlining even across translation units.)
### I only have 1.6.0. Can I still contribute?
Yes, you can! A delta patch is provided to turn a 1.6.0 executable into 1.5.0, so if you are able to dump 1.6.0 from your console you will also be able to get 1.5.0. Read on for more information...
### Do I need to be a game dev or a C++ expert to contribute?
No, of course not. You only need to be familiar with software development practices and C++, or another language with object-oriented aspects such as Java.
Unlike many other decomp projects, familiary with C or assembly helps but is not required. The reason for this is that BotW is a modern game with many layers of abstractions. Many functions can be decompiled simply by recognizing code patterns / inline functions. Our assembly comparing tool ([asm-differ](https://github.com/simonlindholm/asm-differ)) is also capable of showing what each line of assembly corresponds to in the source code, so you won't get lost in the assembly even if you're unfamiliar with AArch64.
AArch64 assembly is also nicer to deal with than x86/x86-64 and should feel familiar to you if you have already worked with e.g. ARM or MIPS.
This is pretty much an open source project, and even small contributions can make a large difference!
While you certainly do not need to be a C++ expert, you do need to be familiar with basic language features and concepts like namespaces or classes. Otherwise, you will be unable to contribute in any efficient or meaningful way.
### Isn't this risky?
Just like other game decompilations, this project is probably in a legal gray zone. However, the authors of this project believe that it is unlikely to bother NCL for the following reasons:
* Contributing to this repository requires owning the game on a Switch console and dumping the executable.
* This project is completely useless to anybody who does not have the game.
* It cannot be used to play the game.
* It does not give you any useful knowledge if you do not play the game or if you do not even have it.
* This repository is only about the executable, which is less than 0.3% of the whole game (ExeFS+RomFS).
* Even if the executable were fully decompiled, it would still not be possible to play the game without dumping the RomFS from a Switch.
* This repository does not contain any original code from the executable.
* Unlike some decompilation projects for older consoles, not even a single byte of assembly code is included from the original executable.
* It only contains reimplemented functions that happen to match once compiled.
* The compiler is Clang, so there are many, many, many ways to write a function and organize things while still getting the exact same code. In fact, while the source code happens to match the compiled code, it is possible and quite likely that the original codebase looks very different from this repository.
* This is a large monolithic game so there is no other way of being accurate to the original logic without doing matching decompilation.
* Clean room decompilation (having separate teams doing reverse engineering and implementation work) is not a solution when the goal is to follow the original logic as accurately as possible.
* This project does not use any proprietary SDK or any leaked document at all.
* The compiler is just Clang 4.0.1 which is open source and freely available on [LLVM's website](https://releases.llvm.org/). The SDK compiler is **not** used.
* Anyone who has had access to leaked information is not allowed to contribute.
### Alright, how do I start contributing?
First, set up the build environment by following the instructions below.
## Building
Reminder: **this will not produce a playable game.** This project will not allow you to play the game if you don't already own it on a Switch.
### For Windows users
While Linux is not a hard requirement, it is strongly advised to [set up WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10) to simplify the setup process. Ubuntu 20.04 is usually a good choice.
The instructions below assume that you are using Linux (native or WSL) or macOS.
### 1. Set up dependencies
* Python 3.6 or newer with [pip](https://pip.pypa.io/en/stable/installation/)
* Ninja
* CMake 3.13+
* If you are on Ubuntu 18.04, you must first [update CMake by using the official CMake APT repository](https://apt.kitware.com/).
* ccache (to speed up builds)
* xdelta3
* Clang (not for compiling BotW code, but for compiling Rust tools)
Ubuntu users can install those dependencies by running:
```shell
sudo apt install python3 ninja-build cmake ccache xdelta3 clang libssl-dev
```
Additionally, you'll also need:
* A Rust toolchain ([follow the instructions here](https://www.rust-lang.org/tools/install))
* The following Python modules: `capstone colorama cxxfilt pyelftools ansiwrap watchdog python-Levenshtein toml` (install them with `pip install ...`)
### 2. Set up the project
1. Clone this repository. If you are using WSL, please clone the repo *inside* WSL, *not* on the Windows side (for performance reasons).
2. Run `git submodule update --init --recursive`
Next, you'll need to acquire the **original 1.5.0 or 1.6.0 `main` NSO executable**.
* To dump it from a Switch, follow [the instructions on the wiki](https://zeldamods.org/wiki/Help:Dumping_games#Dumping_binaries_.28executable_files.29).
* You do not need to dump the entire game (RomFS + ExeFS + DLC). Just dumping the 1.5.0 or 1.6.0 ExeFS is sufficient.
* The decompressed 1.5.0 NSO has the following SHA256 hash: `d9fa308d0ee7c0ab081c66d987523385e1afe06f66731bbfa32628438521c106`
* If you have a compressed NSO or a 1.6.0 executable, don't worry about this.
3. Run `tools/setup.py [path to the NSO]`
* This will:
* install `tools/check` to check for differences in decompiled code
* convert the executable if necessary
* set up [Clang 4.0.1](https://releases.llvm.org/download.html#4.0.1) by downloading it from the official LLVM website
* create a build directory in `build/`
* If something goes wrong, follow the instructions given to you by the script.
### 3. Build
To start the build, just run
```shell
ninja -C build
```
By default, Ninja will perform a multithreaded build. There is no need to pass -j manually.
To check whether everything built correctly, just run `tools/check` after the build completes.
## Contributing
Follow the [contributing guidelines here](Contributing.md). Feel free to join the [Zelda Decompilation](https://discord.zelda64.dev/) Discord server if you have any questions.
## Resources
#### Breath of the Wild documentation and datamining
* The [ZeldaMods](https://zeldamods.org/wiki/Main_Page) wiki
* [MrCheeze's botw-tools](https://github.com/MrCheeze/botw-tools)
* [botw-re-notes](https://github.com/leoetlino/botw-re-notes)
#### AArch64
* AArch64 ISA overview: https://developer.arm.com/documentation/102374/0101 (read this first if you're new to AArch64)
* ARMv8 A64 Quick Reference: https://courses.cs.washington.edu/courses/cse469/18wi/Materials/arm64.pdf
* Arm ARMv8 ARM (Architecture Reference Manual): https://developer.arm.com/documentation/ddi0487/latest/ (extremely detailed)
#### Tools
* IDA Pro + Hex-Rays (7.5+ required if you want to use this project's database/IDC)
* HexRaysPyTools: https://github.com/igogo-x86/HexRaysPyTools
* How to create C++ types and vtables in IDA (7.2+): https://www.hex-rays.com/products/ida/support/idadoc/1691.shtml
* Ghidra: https://ghidra-sre.org/
For more information, see https://botw.link