mirror of
https://github.com/open-goal/jak-project.git
synced 2024-11-23 06:09:57 +00:00
docs: delete content that now lives in another repo (#1160)
This commit is contained in:
Tyler Wilding
GitHub
parent
bf11ef3934
commit
e4c841f9f5
2
.github/dependabot.yml
vendored
2
.github/dependabot.yml
vendored
@ -5,4 +5,4 @@ updates:
|
||||
directory: "/"
|
||||
schedule:
|
||||
# Check for updates to GitHub Actions every week
|
||||
interval: "weekly"
|
||||
interval: "monthly"
|
||||
|
||||
16
README.md
16
README.md
@ -1,5 +1,5 @@
|
||||
<p align="center">
|
||||
<img width="500" height="100%" src="./docs/markdown/imgs/logo-text-colored-new.png">
|
||||
<img width="500" height="100%" src="./docs/img/logo-text-colored-new.png">
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
@ -62,8 +62,8 @@ We have a Discord server where we discuss development. https://discord.gg/V82sTJ
|
||||
So far, we've decompiled around 400,000 lines of GOAL code, out of an estimated 500,000 total lines. We have a working OpenGL renderer which renders most of the game world and foreground. Levels are fully playable, and you can finish the game with 100% completion! There is currently *no* audio.
|
||||
|
||||
Here are some screenshots of the renderer:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
YouTube playlist:
|
||||
https://www.youtube.com/playlist?list=PLWx9T30aAT50cLnCTY1SAbt2TtWQzKfXX
|
||||
@ -74,7 +74,7 @@ We don't save any assets from the game - you must bring your own copy of the gam
|
||||
|
||||
## What's Next
|
||||
|
||||
- Continue decompilation of GOAL code. We've made huge progress recently in decompiling gameplay code. We're finishing that up and also working on the some of the rendering code. Here's our decompilation progress over the past year: 
|
||||
- Continue decompilation of GOAL code. We've made huge progress recently in decompiling gameplay code. We're finishing that up and also working on the some of the rendering code. Here's our decompilation progress over the past year: 
|
||||
- Bug testing! The game can be beaten 100%, but it's possible a few things do not work correctly.
|
||||
- Improve the decompiler. We are always finding new features and macros in the GOAL language.
|
||||
- Investigate more complicated renderers. We have an in-progress port of the "merc" foreground renderer, shown in the screenshots above.
|
||||
@ -182,12 +182,12 @@ git clone https://github.com/open-goal/jak-project.git
|
||||
|
||||
This will create a `jak-project` folder, we will open the project as a CMake project via Visual Studio.
|
||||
|
||||
|
||||
|
||||
|
||||
Then build the entire project as `Windows Release (clang-cl)`. You can also press Ctrl+Shift+B as a hotkey for Build All. We currently prefer `clang-cl` on Windows as opposed to `msvc`, though it should work as well!
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Building and Running the Game
|
||||
|
||||
@ -309,7 +309,7 @@ The first is `goalc`, which is a GOAL compiler for x86-64. Our implementation of
|
||||
|
||||
The second component to the project is the decompiler. You must have a copy of the PS2 game and place all files from the DVD inside a folder corresponding to the game within `iso_data` folder (`jak1` for Jak 1 Black Label, etc.), as seen in this picture:
|
||||
|
||||
|
||||
|
||||
|
||||
Then run `decomp.sh` (Linux) or `decomp-jak1.bat` (Windows) to run the decompiler. The decompiler will extract assets to the `assets` folder. These assets will be used by the compiler when building the port, and you may want to turn asset extraction off after running it once. The decompiler will output code and other data intended to be inspected by humans in the `decompiler_out` folder. Stuff in this folder will not be used by the compiler.
|
||||
|
||||
|
||||
0
docs/.gitignore
vendored
0
docs/.gitignore
vendored
@ -1,98 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/jak-project/favicon.png" />
|
||||
<title>OpenGOAL - API Docs</title>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
|
||||
<meta name="description" content="Description" />
|
||||
<meta
|
||||
name="viewport"
|
||||
content="width=device-width, initial-scale=1.0, minimum-scale=1.0"
|
||||
/>
|
||||
<link
|
||||
rel="stylesheet"
|
||||
href="https://cdn.jsdelivr.net/npm/docsify@4/lib/themes/vue.css"
|
||||
/>
|
||||
<style>
|
||||
.sidebar {
|
||||
background-color: #0d1117;
|
||||
}
|
||||
.sidebar ul li a {
|
||||
color: #c5ccd4;
|
||||
}
|
||||
.sidebar-toggle {
|
||||
background-color: #0d1117;
|
||||
}
|
||||
.app-name-link > img:nth-child(1) {
|
||||
width: 75%;
|
||||
}
|
||||
.search {
|
||||
border-bottom: none !important;
|
||||
}
|
||||
.search a {
|
||||
color: #c5ccd4 !important;
|
||||
}
|
||||
.sidebar li > p {
|
||||
color: #ffe301;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script>
|
||||
window.$docsify = {
|
||||
name: "OpenGOAL Docs",
|
||||
repo: "https://github.com/water111/jak-project",
|
||||
basePath: "./markdown/",
|
||||
loadSidebar: true,
|
||||
subMaxLevel: 3,
|
||||
logo: "./markdown/imgs/logo-text-colored.png",
|
||||
notFoundPage: true,
|
||||
auto2top: true,
|
||||
search: "auto", // default
|
||||
// complete configuration parameters
|
||||
search: {
|
||||
maxAge: 86400000, // Expiration time, the default one day
|
||||
paths: "auto",
|
||||
placeholder: "Search Docs",
|
||||
noData: "No Results!",
|
||||
// Headline depth, 1 - 6
|
||||
depth: 6,
|
||||
hideOtherSidebarContent: false, // whether or not to hide other sidebar content
|
||||
},
|
||||
plugins: [
|
||||
function(hook, vm) {
|
||||
hook.beforeEach(function(html) {
|
||||
if (/githubusercontent\.com/.test(vm.route.file)) {
|
||||
url = vm.route.file
|
||||
.replace('raw.githubusercontent.com', 'github.com')
|
||||
.replace(/\/master/, '/blob/master');
|
||||
} else {
|
||||
url =
|
||||
'https://github.com/water111/jak-project/blob/master/docs' +
|
||||
vm.route.file;
|
||||
}
|
||||
var editHtml = '[:memo: Edit Document](' + url + ')\n';
|
||||
return (
|
||||
editHtml +
|
||||
html
|
||||
);
|
||||
})
|
||||
},
|
||||
],
|
||||
};
|
||||
</script>
|
||||
<!-- Docsify v4 -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/docsify@4"></script>
|
||||
<script src="https://unpkg.com/docsify@4.12.1/lib/plugins/search.min.js"></script>
|
||||
<script src="https://unpkg.com/docsify-copy-code@2"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-bash.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-c.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-cpp.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-clojure.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-lisp.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-scheme.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-nasm.min.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@ -1 +0,0 @@
|
||||
.overlay-page-width[data-v-77084e79]{width:100%}.img-caption[data-v-77084e79]{margin:1em;font-weight:700;text-shadow:-1px -1px 0 #000,1px -1px 0 #000,-1px 1px 0 #000,1px 1px 0 #000}.video-iframe[data-v-77084e79]{min-height:400px}.theme--light.v-overlay{color:rgba(0,0,0,.87)}.theme--dark.v-overlay{color:#fff}.v-overlay{align-items:center;border-radius:inherit;display:flex;justify-content:center;position:fixed;top:0;left:0;right:0;bottom:0;pointer-events:none;transition:.3s cubic-bezier(.25,.8,.5,1),z-index 1ms}.v-overlay__content{position:relative}.v-overlay__scrim{border-radius:inherit;bottom:0;height:100%;left:0;position:absolute;right:0;top:0;transition:inherit;width:100%;will-change:opacity}.v-overlay--absolute{position:absolute}.v-overlay--active{pointer-events:auto}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 6.3 KiB |
BIN
docs/favicon.png
BIN
docs/favicon.png
Binary file not shown.
|
Before Width: | Height: | Size: 4.1 KiB |
26
docs/gh-pages-proj/.gitignore
vendored
26
docs/gh-pages-proj/.gitignore
vendored
@ -1,26 +0,0 @@
|
||||
.DS_Store
|
||||
node_modules
|
||||
/dist
|
||||
|
||||
|
||||
# local env files
|
||||
.env.local
|
||||
.env.*.local
|
||||
|
||||
# Log files
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
|
||||
# Editor directories and files
|
||||
.idea
|
||||
.vscode
|
||||
*.suo
|
||||
*.ntvs*
|
||||
*.njsproj
|
||||
*.sln
|
||||
*.sw?
|
||||
|
||||
#Electron-builder output
|
||||
/dist_electron
|
||||
@ -1,5 +0,0 @@
|
||||
module.exports = {
|
||||
presets: [
|
||||
'@vue/cli-plugin-babel/preset'
|
||||
]
|
||||
}
|
||||
27904
docs/gh-pages-proj/package-lock.json
generated
27904
docs/gh-pages-proj/package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@ -1,56 +0,0 @@
|
||||
{
|
||||
"name": "docs-and-tooling",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"serve": "vue-cli-service serve",
|
||||
"build": "vue-cli-service build",
|
||||
"lint": "vue-cli-service lint"
|
||||
},
|
||||
"main": "main.js",
|
||||
"dependencies": {
|
||||
"axios": "^0.21.1",
|
||||
"core-js": "^3.6.5",
|
||||
"decomment": "^0.9.4",
|
||||
"dotenv": "^8.2.0",
|
||||
"vue": "^2.6.11",
|
||||
"vue-router": "^3.5.1",
|
||||
"vuetify": "^2.4.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@vue/cli-plugin-babel": "^4.5.0",
|
||||
"@vue/cli-plugin-eslint": "^4.5.0",
|
||||
"@vue/cli-plugin-router": "^4.5.12",
|
||||
"@vue/cli-service": "^4.5.0",
|
||||
"babel-eslint": "^10.1.0",
|
||||
"electron": "^11.0.0",
|
||||
"electron-devtools-installer": "^3.1.0",
|
||||
"eslint": "^6.7.2",
|
||||
"eslint-plugin-vue": "^6.2.2",
|
||||
"sass": "^1.32.0",
|
||||
"sass-loader": "^10.0.0",
|
||||
"vue-cli-plugin-vuetify": "^2.3.1",
|
||||
"vue-loader": "^15.9.6",
|
||||
"vue-template-compiler": "^2.6.12",
|
||||
"vuetify-loader": "^1.7.0"
|
||||
},
|
||||
"eslintConfig": {
|
||||
"root": true,
|
||||
"env": {
|
||||
"node": true
|
||||
},
|
||||
"extends": [
|
||||
"plugin:vue/essential",
|
||||
"eslint:recommended"
|
||||
],
|
||||
"parserOptions": {
|
||||
"parser": "babel-eslint"
|
||||
},
|
||||
"rules": {}
|
||||
},
|
||||
"browserslist": [
|
||||
"> 1%",
|
||||
"last 2 versions",
|
||||
"not dead"
|
||||
]
|
||||
}
|
||||
@ -1,98 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" href="/jak-project/favicon.png" />
|
||||
<title>OpenGOAL - API Docs</title>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
|
||||
<meta name="description" content="Description" />
|
||||
<meta
|
||||
name="viewport"
|
||||
content="width=device-width, initial-scale=1.0, minimum-scale=1.0"
|
||||
/>
|
||||
<link
|
||||
rel="stylesheet"
|
||||
href="https://cdn.jsdelivr.net/npm/docsify@4/lib/themes/vue.css"
|
||||
/>
|
||||
<style>
|
||||
.sidebar {
|
||||
background-color: #0d1117;
|
||||
}
|
||||
.sidebar ul li a {
|
||||
color: #c5ccd4;
|
||||
}
|
||||
.sidebar-toggle {
|
||||
background-color: #0d1117;
|
||||
}
|
||||
.app-name-link > img:nth-child(1) {
|
||||
width: 75%;
|
||||
}
|
||||
.search {
|
||||
border-bottom: none !important;
|
||||
}
|
||||
.search a {
|
||||
color: #c5ccd4 !important;
|
||||
}
|
||||
.sidebar li > p {
|
||||
color: #ffe301;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script>
|
||||
window.$docsify = {
|
||||
name: "OpenGOAL Docs",
|
||||
repo: "https://github.com/water111/jak-project",
|
||||
basePath: "./markdown/",
|
||||
loadSidebar: true,
|
||||
subMaxLevel: 3,
|
||||
logo: "./markdown/imgs/logo-text-colored.png",
|
||||
notFoundPage: true,
|
||||
auto2top: true,
|
||||
search: "auto", // default
|
||||
// complete configuration parameters
|
||||
search: {
|
||||
maxAge: 86400000, // Expiration time, the default one day
|
||||
paths: "auto",
|
||||
placeholder: "Search Docs",
|
||||
noData: "No Results!",
|
||||
// Headline depth, 1 - 6
|
||||
depth: 6,
|
||||
hideOtherSidebarContent: false, // whether or not to hide other sidebar content
|
||||
},
|
||||
plugins: [
|
||||
function(hook, vm) {
|
||||
hook.beforeEach(function(html) {
|
||||
if (/githubusercontent\.com/.test(vm.route.file)) {
|
||||
url = vm.route.file
|
||||
.replace('raw.githubusercontent.com', 'github.com')
|
||||
.replace(/\/master/, '/blob/master');
|
||||
} else {
|
||||
url =
|
||||
'https://github.com/water111/jak-project/blob/master/docs' +
|
||||
vm.route.file;
|
||||
}
|
||||
var editHtml = '[:memo: Edit Document](' + url + ')\n';
|
||||
return (
|
||||
editHtml +
|
||||
html
|
||||
);
|
||||
})
|
||||
},
|
||||
],
|
||||
};
|
||||
</script>
|
||||
<!-- Docsify v4 -->
|
||||
<script src="https://cdn.jsdelivr.net/npm/docsify@4"></script>
|
||||
<script src="https://unpkg.com/docsify@4.12.1/lib/plugins/search.min.js"></script>
|
||||
<script src="https://unpkg.com/docsify-copy-code@2"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-bash.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-c.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-cpp.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-clojure.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-lisp.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-scheme.min.js"></script>
|
||||
<script src="//cdn.jsdelivr.net/npm/prismjs@1/components/prism-nasm.min.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 4.1 KiB |
@ -1,32 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1.0">
|
||||
<link rel="icon" href="<%= BASE_URL %>favicon.png">
|
||||
<title>OpenGOAL - The Jak and Daxter Project</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Fira+Code&family=Lexend:wght@400;700&display=swap" rel="stylesheet">
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@mdi/font@latest/css/materialdesignicons.min.css">
|
||||
</head>
|
||||
<style>
|
||||
body {
|
||||
font-family: 'Lexend', sans-serif !important;
|
||||
}
|
||||
code {
|
||||
font-family: 'Fira Code', monospace !important;
|
||||
}
|
||||
.v-application {
|
||||
font-family: 'Lexend', sans-serif !important;
|
||||
}
|
||||
</style>
|
||||
<body>
|
||||
<noscript>
|
||||
<strong>We're sorry but <%= htmlWebpackPlugin.options.title %> doesn't work properly without JavaScript enabled. Please enable it to continue.</strong>
|
||||
</noscript>
|
||||
<div id="app"></div>
|
||||
<!-- built files will be auto injected -->
|
||||
</body>
|
||||
</html>
|
||||
@ -1,17 +0,0 @@
|
||||
<template>
|
||||
<v-app>
|
||||
<v-main>
|
||||
<router-view />
|
||||
</v-main>
|
||||
</v-app>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
export default {
|
||||
name: "App",
|
||||
components: {},
|
||||
data: () => ({
|
||||
//
|
||||
})
|
||||
};
|
||||
</script>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.0 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 322 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 634 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.3 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.2 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 3.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.9 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.4 MiB |
@ -1,10 +0,0 @@
|
||||
[
|
||||
{
|
||||
"link": "https://www.youtube.com/embed/ZO7A22btJc0",
|
||||
"timestamp": "2022-02-04"
|
||||
},
|
||||
{
|
||||
"link": "https://www.youtube.com/watch?v=6dJhyVqqq5Q",
|
||||
"timestamp": "2022-01-15"
|
||||
}
|
||||
]
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 993 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 359 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 104 KiB |
@ -1,93 +0,0 @@
|
||||
{
|
||||
"jak1": {
|
||||
"name": "Jak 1",
|
||||
"media": [
|
||||
{
|
||||
"fileName": "robotboss-appears_2022-02-04.png",
|
||||
"timestamp": "2022-02-04",
|
||||
"caption": "Robotboss Appears",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "lurker-chilling_2022-02-04.png",
|
||||
"timestamp": "2022-02-04",
|
||||
"caption": "Lurker Chilling",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "jak-and-daxter-are-stunned_2022-02-04.png",
|
||||
"timestamp": "2022-02-04",
|
||||
"caption": "Jak And Daxter Are Stunned",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"link": "https://www.youtube.com/embed/ZO7A22btJc0",
|
||||
"timestamp": "2022-02-04",
|
||||
"video": true
|
||||
},
|
||||
{
|
||||
"fileName": "light-cave_2022-02-02.png",
|
||||
"timestamp": "2022-02-02",
|
||||
"caption": "Light Cave",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "entity-debugging_2022-01-31.png",
|
||||
"timestamp": "2022-01-31",
|
||||
"caption": "Entity Debugging",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "bones-see-light_2022-01-28.png",
|
||||
"timestamp": "2022-01-28",
|
||||
"caption": "Bones See Light",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"link": "https://www.youtube.com/embed/6dJhyVqqq5Q",
|
||||
"timestamp": "2022-01-15",
|
||||
"video": true
|
||||
},
|
||||
{
|
||||
"fileName": "can-save-and-load_2022-01-11.png",
|
||||
"timestamp": "2022-01-11",
|
||||
"caption": "Can Save And Load",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "starry-sky_2022-01-11.png",
|
||||
"timestamp": "2022-01-11",
|
||||
"caption": "Starry Sky",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "boggy-reflections_2022-01-10.png",
|
||||
"timestamp": "2022-01-10",
|
||||
"caption": "Boggy Reflections",
|
||||
"video": false
|
||||
},
|
||||
{
|
||||
"fileName": "too-much-fog_2022-01-10.png",
|
||||
"timestamp": "2022-01-10",
|
||||
"caption": "Too Much Fog",
|
||||
"video": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"jak2": {
|
||||
"name": "Jak 2",
|
||||
"media": []
|
||||
},
|
||||
"jak3": {
|
||||
"name": "Jak 3",
|
||||
"media": []
|
||||
},
|
||||
"jakx": {
|
||||
"name": "Jak X",
|
||||
"media": []
|
||||
},
|
||||
"misc": {
|
||||
"name": "Miscellaneous",
|
||||
"media": []
|
||||
}
|
||||
}
|
||||
@ -1,9 +0,0 @@
|
||||
{
|
||||
"jak1": {
|
||||
"fileProgress": {
|
||||
"src_files_total": 523,
|
||||
"src_files_finished": 190,
|
||||
"src_files_started": 314
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,3 +0,0 @@
|
||||
const galleryLinks = require('./config/gallery.json')
|
||||
|
||||
export default galleryLinks
|
||||
@ -1,12 +0,0 @@
|
||||
import Vue from 'vue'
|
||||
import App from './App.vue'
|
||||
import vuetify from './plugins/vuetify';
|
||||
import router from './router'
|
||||
|
||||
Vue.config.productionTip = false
|
||||
|
||||
new Vue({
|
||||
vuetify,
|
||||
router,
|
||||
render: h => h(App)
|
||||
}).$mount('#app')
|
||||
@ -1,8 +0,0 @@
|
||||
import Vue from 'vue';
|
||||
import Vuetify from 'vuetify/lib/framework';
|
||||
|
||||
Vue.use(Vuetify);
|
||||
|
||||
export default new Vuetify({
|
||||
theme: { dark: true },
|
||||
})
|
||||
@ -1,3 +0,0 @@
|
||||
const projectProgress = require('./config/progress.json')
|
||||
|
||||
export default projectProgress
|
||||
@ -1,25 +0,0 @@
|
||||
import Vue from 'vue'
|
||||
import VueRouter from 'vue-router'
|
||||
|
||||
Vue.use(VueRouter)
|
||||
|
||||
const routes = [
|
||||
{
|
||||
path: '/',
|
||||
name: 'Home',
|
||||
component: () => import('../views/Home.vue')
|
||||
},
|
||||
{
|
||||
path: '/gallery',
|
||||
name: 'Gallery',
|
||||
component: () => import('../views/Gallery.vue')
|
||||
}
|
||||
]
|
||||
|
||||
const router = new VueRouter({
|
||||
mode: 'history',
|
||||
base: process.env.BASE_URL,
|
||||
routes
|
||||
})
|
||||
|
||||
export default router
|
||||
@ -1,138 +0,0 @@
|
||||
<template>
|
||||
<v-container fluid>
|
||||
<v-row v-for="(game, name) in assetLinks" :key="name">
|
||||
<v-col cols="12" align="center">
|
||||
<h2>{{ sectionMetadata[name].name }}</h2>
|
||||
</v-col>
|
||||
<v-col
|
||||
v-for="(asset, index) in game"
|
||||
:key="name + asset + index"
|
||||
class="d-flex child-flex"
|
||||
cols="12"
|
||||
md="4"
|
||||
>
|
||||
<v-img
|
||||
v-if="!asset.video"
|
||||
:src="require(`@/assets/gallery/${name}/${asset.fileName}`)"
|
||||
:lazy-src="require(`@/assets/gallery/${name}/${asset.fileName}`)"
|
||||
aspect-ratio="1"
|
||||
class="grey lighten-2"
|
||||
@click="assetSelected(name, asset)"
|
||||
>
|
||||
<template v-slot:placeholder>
|
||||
<v-row class="fill-height ma-0" align="center" justify="center">
|
||||
<v-progress-circular
|
||||
indeterminate
|
||||
color="grey lighten-5"
|
||||
></v-progress-circular>
|
||||
</v-row>
|
||||
</template>
|
||||
</v-img>
|
||||
<iframe
|
||||
v-if="asset.video"
|
||||
class="video-iframe"
|
||||
:src="asset.link"
|
||||
frameBorder="0"
|
||||
></iframe>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-overlay class="overlay-page-width" v-if="selectedItem.asset">
|
||||
<v-container>
|
||||
<v-row>
|
||||
<v-img
|
||||
:src="
|
||||
require(`@/assets/gallery/${selectedItem.sectionName}/${selectedItem.asset.fileName}`)
|
||||
"
|
||||
:lazy-src="
|
||||
require(`@/assets/gallery/${selectedItem.sectionName}/${selectedItem.asset.fileName}`)
|
||||
"
|
||||
contain
|
||||
@click="assetDeselected()"
|
||||
></v-img>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<h3 class="img-caption">{{ selectedItem.asset.caption }}</h3>
|
||||
</v-row>
|
||||
</v-container>
|
||||
</v-overlay>
|
||||
</v-container>
|
||||
</template>
|
||||
<style scoped>
|
||||
.overlay-page-width {
|
||||
width: 100%;
|
||||
}
|
||||
.img-caption {
|
||||
margin: 1em;
|
||||
font-weight: 700;
|
||||
text-shadow: -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000,
|
||||
1px 1px 0 #000;
|
||||
}
|
||||
.video-iframe {
|
||||
min-height: 400px;
|
||||
}
|
||||
</style>
|
||||
<script>
|
||||
import galleryLinks from "../gallery";
|
||||
|
||||
export default {
|
||||
name: "Gallery",
|
||||
components: {},
|
||||
data: function() {
|
||||
return {
|
||||
selectedItem: {
|
||||
sectionName: "",
|
||||
asset: null
|
||||
},
|
||||
sectionMetadata: {
|
||||
jak1: {
|
||||
name: ""
|
||||
},
|
||||
jak2: {
|
||||
name: ""
|
||||
},
|
||||
jak3: {
|
||||
name: ""
|
||||
},
|
||||
jakx: {
|
||||
name: ""
|
||||
},
|
||||
misc: {
|
||||
name: ""
|
||||
}
|
||||
},
|
||||
assetLinks: {
|
||||
jak1: [],
|
||||
jak2: [],
|
||||
jak3: [],
|
||||
jakx: [],
|
||||
misc: []
|
||||
}
|
||||
};
|
||||
},
|
||||
mounted: async function() {
|
||||
const keys = Object.keys(galleryLinks);
|
||||
for (let i = 0; i < keys.length; i++) {
|
||||
const key = keys[i];
|
||||
this.sectionMetadata[key].name = galleryLinks[key]["name"];
|
||||
for (let j = 0; j < galleryLinks[key]["media"].length; j++) {
|
||||
const asset = galleryLinks[key]["media"][j];
|
||||
this.assetLinks[key].push(asset);
|
||||
}
|
||||
}
|
||||
},
|
||||
methods: {
|
||||
assetSelected: function(sectionName, asset) {
|
||||
this.selectedItem = {
|
||||
sectionName: sectionName,
|
||||
asset: asset
|
||||
};
|
||||
},
|
||||
assetDeselected: function() {
|
||||
this.selectedItem = {
|
||||
sectionName: "",
|
||||
asset: ""
|
||||
};
|
||||
}
|
||||
}
|
||||
};
|
||||
</script>
|
||||
@ -1,299 +0,0 @@
|
||||
<template>
|
||||
<v-container fluid>
|
||||
<v-row justify="center" align="center" class="bg-img">
|
||||
<v-col cols="12" align="center">
|
||||
<v-img
|
||||
max-height="150"
|
||||
max-width="250"
|
||||
src="~@/assets/img/logo-text-colored.png"
|
||||
>
|
||||
</v-img>
|
||||
<br />
|
||||
<h4 class="text-stroke">
|
||||
Reviving the Language that Brought us the Jak & Daxter Series
|
||||
</h4>
|
||||
<br />
|
||||
<v-row justify="center">
|
||||
<v-col cols="auto">
|
||||
<v-btn href="#project-status" rounded color="pink darken-4">
|
||||
<v-icon>mdi-calendar-check</v-icon>
|
||||
Project Status
|
||||
</v-btn>
|
||||
</v-col>
|
||||
<v-col cols="auto">
|
||||
<v-btn to="/gallery" rounded color="green darken-1">
|
||||
<v-icon>mdi-image</v-icon>
|
||||
Gallery
|
||||
</v-btn>
|
||||
</v-col>
|
||||
<v-col cols="auto">
|
||||
<v-btn
|
||||
href="/jak-project/api-docs.html"
|
||||
target="_blank"
|
||||
rounded
|
||||
color="indigo darken-1"
|
||||
>
|
||||
<v-icon>mdi-file-document</v-icon>
|
||||
Documentation
|
||||
</v-btn>
|
||||
</v-col>
|
||||
<v-col cols="auto">
|
||||
<v-btn
|
||||
href="https://github.com/water111/jak-project"
|
||||
target="_blank"
|
||||
rounded
|
||||
color="deep-purple darken-1"
|
||||
>
|
||||
<v-icon>mdi-git</v-icon>
|
||||
Contribute
|
||||
</v-btn>
|
||||
</v-col>
|
||||
</v-row>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row style="margin-top: 3em;">
|
||||
<v-col
|
||||
align="center"
|
||||
justify="center"
|
||||
cols="12"
|
||||
id="project-status"
|
||||
class="orange--text text--darken-1"
|
||||
>
|
||||
<h2>Project Status</h2>
|
||||
</v-col>
|
||||
<v-container style="margin-top: 2em;">
|
||||
<v-row>
|
||||
<v-col cols="12" md="4">
|
||||
<v-row>
|
||||
<v-col
|
||||
align="center"
|
||||
justify="center"
|
||||
class="orange--text text--darken-2"
|
||||
>
|
||||
<h3>Progress Tracker</h3>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col
|
||||
align="center"
|
||||
justify="center"
|
||||
class="orange--text text--darken-3"
|
||||
>
|
||||
<h4>Jak 1 - Black Label - NTSC</h4>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col
|
||||
align="center"
|
||||
justify="center"
|
||||
class="orange--text text--darken-4"
|
||||
>
|
||||
<h5>Decompilation</h5>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col align="center" justify="center">
|
||||
<v-icon class="green--text">mdi-check</v-icon>
|
||||
Files Finished - {{ jak1BlackLabelStatus.srcFilesFinished }} /
|
||||
{{ jak1BlackLabelStatus.srcFilesTotal }}
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col align="center" justify="center">
|
||||
<v-icon class="yellow--text">mdi-timer-outline</v-icon>
|
||||
Files In Progress - {{ jak1BlackLabelStatus.srcFilesStarted }} /
|
||||
{{ jak1BlackLabelStatus.srcFilesTotal }}
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col
|
||||
align="center"
|
||||
justify="center"
|
||||
class="orange--text text--darken-4"
|
||||
>
|
||||
<h5>Renderers and Core Pieces</h5>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col
|
||||
v-for="(milestone, index) in majorMilestones.jak1"
|
||||
:key="'jak1-milestone' + index"
|
||||
cols="12"
|
||||
md="6"
|
||||
align="center"
|
||||
justify="center"
|
||||
>
|
||||
<v-icon
|
||||
v-if="milestone.status === 'Completed'"
|
||||
class="green--text"
|
||||
>
|
||||
mdi-check
|
||||
</v-icon>
|
||||
<v-icon v-else class="yellow--text">
|
||||
mdi-timer-outline
|
||||
</v-icon>
|
||||
{{ milestone.name }}
|
||||
</v-col>
|
||||
</v-row>
|
||||
</v-col>
|
||||
<v-col cols="12" md="8">
|
||||
<v-row>
|
||||
<v-col
|
||||
align="center"
|
||||
justify="center"
|
||||
class="orange--text text--darken-2"
|
||||
>
|
||||
<h3>GitHub Updates</h3>
|
||||
</v-col>
|
||||
</v-row>
|
||||
<v-row>
|
||||
<v-col>
|
||||
<v-timeline dense>
|
||||
<v-timeline-item
|
||||
v-for="(pr, index) in recentPRs"
|
||||
:key="'pr' + index"
|
||||
>
|
||||
<template v-slot:icon>
|
||||
<v-avatar>
|
||||
<img :src="pr.user.avatar_url" />
|
||||
</v-avatar>
|
||||
</template>
|
||||
<template v-slot:opposite>
|
||||
<span>{{ pr.user.login }}</span>
|
||||
</template>
|
||||
<v-card class="elevation-2">
|
||||
<v-card-title>
|
||||
<h5>{{ pr.title }}</h5>
|
||||
</v-card-title>
|
||||
<v-card-text>
|
||||
{{ pr.body }}
|
||||
</v-card-text>
|
||||
<v-card-actions>
|
||||
<v-btn
|
||||
text
|
||||
color="accent"
|
||||
:href="pr.html_url"
|
||||
target="_blank"
|
||||
>
|
||||
View Change
|
||||
</v-btn>
|
||||
</v-card-actions>
|
||||
</v-card>
|
||||
</v-timeline-item>
|
||||
</v-timeline>
|
||||
</v-col>
|
||||
</v-row>
|
||||
</v-col>
|
||||
</v-row>
|
||||
</v-container>
|
||||
</v-row>
|
||||
</v-container>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.bg-img {
|
||||
background-image: url("~@/assets/img/background-new.jpg");
|
||||
background-repeat: no-repeat;
|
||||
background-position: center;
|
||||
background-size: cover;
|
||||
min-height: 100vh;
|
||||
}
|
||||
.wrapped-pre {
|
||||
word-wrap: normal;
|
||||
white-space: pre-wrap;
|
||||
font-family: "Lexend", sans-serif !important;
|
||||
}
|
||||
.text-stroke {
|
||||
text-shadow: -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000,
|
||||
1px 1px 0 #000;
|
||||
}
|
||||
</style>
|
||||
|
||||
<script>
|
||||
import projectProgress from "../progress";
|
||||
|
||||
export default {
|
||||
name: "Home",
|
||||
components: {},
|
||||
data: function() {
|
||||
return {
|
||||
recentPRs: [],
|
||||
majorMilestones: {
|
||||
jak1: [
|
||||
{
|
||||
name: "Sky",
|
||||
status: "Completed"
|
||||
},
|
||||
{
|
||||
name: "TFrag",
|
||||
status: "Completed"
|
||||
},
|
||||
{
|
||||
name: "TIE",
|
||||
status: "Completed"
|
||||
},
|
||||
{
|
||||
name: "Shrub",
|
||||
status: "In-Progress"
|
||||
},
|
||||
{
|
||||
name: "Ocean",
|
||||
status: "In-Progress"
|
||||
},
|
||||
{
|
||||
name: "MERC",
|
||||
status: "In-Progress"
|
||||
},
|
||||
{
|
||||
name: "Shadow",
|
||||
status: "In-Progress"
|
||||
},
|
||||
{
|
||||
name: "Generic",
|
||||
status: "In-Progress"
|
||||
},
|
||||
{
|
||||
name: "Collision",
|
||||
status: "In-Progress"
|
||||
},
|
||||
{
|
||||
name: "Bones",
|
||||
status: "In-Progress"
|
||||
}
|
||||
]
|
||||
},
|
||||
jak1BlackLabelStatus: {
|
||||
srcFilesTotal: projectProgress.jak1.fileProgress.src_files_total,
|
||||
srcFilesFinished: projectProgress.jak1.fileProgress.src_files_finished,
|
||||
srcFilesStarted: projectProgress.jak1.fileProgress.src_files_started
|
||||
}
|
||||
};
|
||||
},
|
||||
mounted: async function() {
|
||||
await this.loadRecentPRs();
|
||||
},
|
||||
methods: {
|
||||
truncateString: function(str, num) {
|
||||
if (str.length <= num) {
|
||||
return str;
|
||||
}
|
||||
return str.slice(0, num) + "...";
|
||||
},
|
||||
loadRecentPRs: async function() {
|
||||
const response = await fetch(
|
||||
`https://api.github.com/search/issues?q=repo:water111/jak-project+is:pr+is:merged&sort=updated`
|
||||
);
|
||||
const data = await response.json();
|
||||
const numPRs = 25;
|
||||
for (var i = 0; i < numPRs; i++) {
|
||||
var pr = data.items[i];
|
||||
if (pr.body == null || pr.body.length == 0) {
|
||||
pr.body = "No Description";
|
||||
}
|
||||
pr.body = this.truncateString(pr.body, 250);
|
||||
this.recentPRs.push(pr);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
</script>
|
||||
@ -1,9 +0,0 @@
|
||||
module.exports = {
|
||||
transpileDependencies: [
|
||||
'vuetify'
|
||||
],
|
||||
filenameHashing: false,
|
||||
publicPath: process.env.NODE_ENV === 'production'
|
||||
? '/jak-project/'
|
||||
: '/'
|
||||
}
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 993 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.0 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 322 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 634 KiB |
|
Before Width: | Height: | Size: 17 KiB After Width: | Height: | Size: 17 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.3 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.2 MiB |
|
Before Width: | Height: | Size: 104 KiB After Width: | Height: | Size: 104 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 104 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 3.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.9 MiB |
|
Before Width: | Height: | Size: 718 KiB After Width: | Height: | Size: 718 KiB |
|
Before Width: | Height: | Size: 754 KiB After Width: | Height: | Size: 754 KiB |
|
Before Width: | Height: | Size: 898 KiB After Width: | Height: | Size: 898 KiB |
|
Before Width: | Height: | Size: 738 KiB After Width: | Height: | Size: 738 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.4 MiB |
|
Before Width: | Height: | Size: 9.9 KiB After Width: | Height: | Size: 9.9 KiB |
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
|
Before Width: | Height: | Size: 7.5 KiB After Width: | Height: | Size: 7.5 KiB |
@ -1,9 +0,0 @@
|
||||
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" href="/jak-project/favicon.png"><title>OpenGOAL - The Jak and Daxter Project</title><link rel="preconnect" href="https://fonts.googleapis.com"><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin><link href="https://fonts.googleapis.com/css2?family=Fira+Code&family=Lexend:wght@400;700&display=swap" rel="stylesheet"><link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@mdi/font@latest/css/materialdesignicons.min.css"><link href="/jak-project/css/chunk-0406d06d.css" rel="prefetch"><link href="/jak-project/css/chunk-461a09f2.css" rel="prefetch"><link href="/jak-project/css/chunk-a7b0a1b0.css" rel="prefetch"><link href="/jak-project/js/chunk-0406d06d.js" rel="prefetch"><link href="/jak-project/js/chunk-461a09f2.js" rel="prefetch"><link href="/jak-project/js/chunk-a7b0a1b0.js" rel="prefetch"><link href="/jak-project/css/chunk-vendors.css" rel="preload" as="style"><link href="/jak-project/js/app.js" rel="preload" as="script"><link href="/jak-project/js/chunk-vendors.js" rel="preload" as="script"><link href="/jak-project/css/chunk-vendors.css" rel="stylesheet"></head><style>body {
|
||||
font-family: 'Lexend', sans-serif !important;
|
||||
}
|
||||
code {
|
||||
font-family: 'Fira Code', monospace !important;
|
||||
}
|
||||
.v-application {
|
||||
font-family: 'Lexend', sans-serif !important;
|
||||
}</style><body><noscript><strong>We're sorry but docs-and-tooling doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript><div id="app"></div><script src="/jak-project/js/chunk-vendors.js"></script><script src="/jak-project/js/app.js"></script></body></html>
|
||||
@ -1,2 +0,0 @@
|
||||
(function(e){function t(t){for(var r,o,c=t[0],i=t[1],l=t[2],s=0,p=[];s<c.length;s++)o=c[s],Object.prototype.hasOwnProperty.call(a,o)&&a[o]&&p.push(a[o][0]),a[o]=0;for(r in i)Object.prototype.hasOwnProperty.call(i,r)&&(e[r]=i[r]);f&&f(t);while(p.length)p.shift()();return u.push.apply(u,l||[]),n()}function n(){for(var e,t=0;t<u.length;t++){for(var n=u[t],r=!0,o=1;o<n.length;o++){var c=n[o];0!==a[c]&&(r=!1)}r&&(u.splice(t--,1),e=i(i.s=n[0]))}return e}var r={},o={app:0},a={app:0},u=[];function c(e){return i.p+"js/"+({}[e]||e)+".js"}function i(t){if(r[t])return r[t].exports;var n=r[t]={i:t,l:!1,exports:{}};return e[t].call(n.exports,n,n.exports,i),n.l=!0,n.exports}i.e=function(e){var t=[],n={"chunk-a7b0a1b0":1,"chunk-0406d06d":1,"chunk-461a09f2":1};o[e]?t.push(o[e]):0!==o[e]&&n[e]&&t.push(o[e]=new Promise((function(t,n){for(var r="css/"+({}[e]||e)+".css",a=i.p+r,u=document.getElementsByTagName("link"),c=0;c<u.length;c++){var l=u[c],s=l.getAttribute("data-href")||l.getAttribute("href");if("stylesheet"===l.rel&&(s===r||s===a))return t()}var p=document.getElementsByTagName("style");for(c=0;c<p.length;c++){l=p[c],s=l.getAttribute("data-href");if(s===r||s===a)return t()}var f=document.createElement("link");f.rel="stylesheet",f.type="text/css",f.onload=t,f.onerror=function(t){var r=t&&t.target&&t.target.src||a,u=new Error("Loading CSS chunk "+e+" failed.\n("+r+")");u.code="CSS_CHUNK_LOAD_FAILED",u.request=r,delete o[e],f.parentNode.removeChild(f),n(u)},f.href=a;var d=document.getElementsByTagName("head")[0];d.appendChild(f)})).then((function(){o[e]=0})));var r=a[e];if(0!==r)if(r)t.push(r[2]);else{var u=new Promise((function(t,n){r=a[e]=[t,n]}));t.push(r[2]=u);var l,s=document.createElement("script");s.charset="utf-8",s.timeout=120,i.nc&&s.setAttribute("nonce",i.nc),s.src=c(e);var p=new Error;l=function(t){s.onerror=s.onload=null,clearTimeout(f);var n=a[e];if(0!==n){if(n){var r=t&&("load"===t.type?"missing":t.type),o=t&&t.target&&t.target.src;p.message="Loading chunk "+e+" failed.\n("+r+": "+o+")",p.name="ChunkLoadError",p.type=r,p.request=o,n[1](p)}a[e]=void 0}};var f=setTimeout((function(){l({type:"timeout",target:s})}),12e4);s.onerror=s.onload=l,document.head.appendChild(s)}return Promise.all(t)},i.m=e,i.c=r,i.d=function(e,t,n){i.o(e,t)||Object.defineProperty(e,t,{enumerable:!0,get:n})},i.r=function(e){"undefined"!==typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},i.t=function(e,t){if(1&t&&(e=i(e)),8&t)return e;if(4&t&&"object"===typeof e&&e&&e.__esModule)return e;var n=Object.create(null);if(i.r(n),Object.defineProperty(n,"default",{enumerable:!0,value:e}),2&t&&"string"!=typeof e)for(var r in e)i.d(n,r,function(t){return e[t]}.bind(null,r));return n},i.n=function(e){var t=e&&e.__esModule?function(){return e["default"]}:function(){return e};return i.d(t,"a",t),t},i.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)},i.p="/jak-project/",i.oe=function(e){throw console.error(e),e};var l=window["webpackJsonp"]=window["webpackJsonp"]||[],s=l.push.bind(l);l.push=t,l=l.slice();for(var p=0;p<l.length;p++)t(l[p]);var f=s;u.push([0,"chunk-vendors"]),n()})({0:function(e,t,n){e.exports=n("56d7")},"56d7":function(e,t,n){"use strict";n.r(t);n("e260"),n("e6cf"),n("cca6"),n("a79d");var r=n("2b0e"),o=function(){var e=this,t=e.$createElement,n=e._self._c||t;return n("v-app",[n("v-main",[n("router-view")],1)],1)},a=[],u={name:"App",components:{},data:function(){return{}}},c=u,i=n("2877"),l=n("6544"),s=n.n(l),p=n("7496"),f=n("f6c4"),d=Object(i["a"])(c,o,a,!1,null,null,null),h=d.exports;s()(d,{VApp:p["a"],VMain:f["a"]});var m=n("f309");r["a"].use(m["a"]);var v=new m["a"]({theme:{dark:!0}}),b=(n("d3b7"),n("3ca3"),n("ddb0"),n("8c4f"));r["a"].use(b["a"]);var g=[{path:"/",name:"Home",component:function(){return Promise.all([n.e("chunk-a7b0a1b0"),n.e("chunk-461a09f2")]).then(n.bind(null,"bb51"))}},{path:"/gallery",name:"Gallery",component:function(){return Promise.all([n.e("chunk-a7b0a1b0"),n.e("chunk-0406d06d")]).then(n.bind(null,"0d77"))}}],y=new b["a"]({mode:"history",base:"/jak-project/",routes:g}),w=y;r["a"].config.productionTip=!1,new r["a"]({vuetify:v,router:w,render:function(e){return e(h)}}).$mount("#app")}});
|
||||
//# sourceMappingURL=app.js.map
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 104 KiB |
@ -1,37 +0,0 @@
|
||||
# An Overview
|
||||
|
||||
This is the main documentation for the OpenGOAL language. It's designed to be read in order to learn OpenGOAL. **It does not explain the OpenGOAL kernel or state system.**
|
||||
|
||||
The syntax descriptions uses these rules:
|
||||
- Something `[in-brackets]` is optional and can be left out.
|
||||
- Something like `[:type type-name]` means there is an optional named argument.
|
||||
- It can be used like `:type type-name`, replacing `type-name` with what you want, or left out entirely.
|
||||
- When there are multiple choices, they are separated by `|`. Example: `#t|#f` is either `#t` or `#f`.
|
||||
- `...` means more of the thing before can be included. Example `(f arg...)` can have multiple arguments.
|
||||
|
||||
## Language Basics
|
||||
|
||||
OpenGOAL is a compiled language. Source code is stored in `.gc` files. Each `.gc` file is compiled into a `.o` file. These `.o` files are then loaded by the game. When they are loaded, it has the effect of running every "top level" expression in the file. Usually these are function, type, and method declarations, but you can also use this for initialization code. For example, it is common to first define types, functions, and methods, then set up global instances.
|
||||
|
||||
There are effectively three different "languages":
|
||||
1. **OpenGOAL** - the normal compiled language.
|
||||
2. **OpenGOAL compiler commands** - simple commands to run the compiler, listener, and debugger. These run in the compiler only.
|
||||
3. **GOOS** macro language. This is used in OpenGOAL macros and runs at compile-time. These macros generate OpenGOAL compiler commands or OpenGOAL source which is then processed. These run in the compiler only.
|
||||
|
||||
The OpenGOAL language uses a LISP syntax, but on the inside is closer to C or C++. There is no protection against use-after-free or other common pointer bugs.
|
||||
|
||||
Unlike a C/C++ compiler, the OpenGOAL compiler has a state. It remembers functions/methods/types/macros/constants/enums/etc defined in previous files.
|
||||
|
||||
## Type System Introduction
|
||||
|
||||
OpenGOAL has a type system. Every expression and object in OpenGOAL has a type. With the exception of three special types (`none`, `_varags_`, and `_types_`), every type has a parent type, and the root of all types is `object`. Types themselves are objects in the runtime that contain some basic information and their method table.
|
||||
|
||||
One annoying detail of OpenGOAL is that the type system has some slightly different types for variables and places in memory, and automatic conversions between them.
|
||||
|
||||
Another annoying detail is that there are a totally separate set of rules for 128-bit integer types (or children of these). Nothing in this section applies to these.
|
||||
|
||||
Some types are boxed vs. unboxed. If you have an object of boxed type, it is possible to figure out its type at runtime. If you have an object of unboxed type, you can't. If you have an unboxed type, you can't tell if it's a boxed or unboxed object.
|
||||
|
||||
Some types are value or reference. A value type means it has value semantics, it is passed by value everywhere. A reference type is like a C/C++ pointer or reference, where there is memory allocated for the data somewhere, and the you just pass around a reference to this memory.
|
||||
|
||||
For more information on the type system, see [the following](type_system.md)
|
||||
@ -1 +0,0 @@
|
||||
# 404 - Doc Page not Found!
|
||||
@ -1,27 +0,0 @@
|
||||
- OpenGOAL Reference
|
||||
- [Overview](/README.md)
|
||||
- [Type System](/type_system.md)
|
||||
- [Method System](/method_system.md)
|
||||
- [Language Syntax & Features](/syntax.md)
|
||||
- [Standard Library](/lib.md)
|
||||
- [The Reader](/reader.md)
|
||||
- [Macro Support](/goos.md)
|
||||
- [Object File Formats](/object_file_formats.md)
|
||||
- [Process and State System](/process_and_state.md)
|
||||
- [States in the Decompiler](/decompiler_states.md)
|
||||
|
||||
- Working with OpenGOAL
|
||||
- [The REPL](/repl.md)
|
||||
- [Debugging](/debugging.md)
|
||||
- [Editor Configuration](/editor_setup.md)
|
||||
|
||||
- Developing OpenGOAL
|
||||
- [Compiler Walkthrough](/compiler_example.md)
|
||||
- [Assembly Emitter](/asm_emitter.md)
|
||||
- [Porting to x86](/porting_to_x86.md)
|
||||
- [Register Handling](/registers.md)
|
||||
|
||||
- PC Port Documentation
|
||||
- [Graphics](/graphics.md)
|
||||
- [Drawable and TFRAG](/drawable_and_tfrag.md)
|
||||
- [Porting TFRAG](/porting_tfrag.md)
|
||||
@ -1,25 +0,0 @@
|
||||
# Assembly Emitter
|
||||
|
||||
x86-64 has a lot of instructions. They are described in Volume 2 of the 5 Volume "Intel® 64 and IA-32 Architectures Software Developer’s Manual". Just this volume alone is over 2000 pages, which would take forever to fully implement. As a result, we will use only a subset of these instructions. This the rough plan:
|
||||
|
||||
- Most instructions like `add` will only be implemented with `r64 r64` versions.
|
||||
- To accomplish something like `add rax, 1`, we will use a temporary register `X`
|
||||
- `mov X, 1`
|
||||
- `add rax, X`
|
||||
- The constant propagation system will be able to provide enough information that we could eventually use `add r64 immX` and similar if needed.
|
||||
- Register allocation should handle the case `(set! x (+ 3 y))` as:
|
||||
- `mov x, 3`
|
||||
- `add x, y`
|
||||
- but `(set! x (+ y 3))`, in cases where `y` is needed after and `x` can't take its place, will become the inefficient
|
||||
- `mov x, y`
|
||||
- `mov rtemp, 3`
|
||||
- `add x, rtemp`
|
||||
- Loading constants into registers will be done efficiently, using the same strategy used by modern versions of `gcc` and `clang`.
|
||||
- Memory access will be done in the form `mov rdest, [roff + raddr]` where `roff` is the offset register. Doing memory access in this form was found to be much faster in simple benchmark test.
|
||||
- Memory access to the stack will have an extra `sub` and more complicated dereference. GOAL code seems to avoid using the stack in most places, and I suspect the programmers attempted to avoid stack spills.
|
||||
- `mov rdest, rsp` : coloring move for upcoming subtract
|
||||
- `sub rdest, roff` : convert real pointer to GOAL pointer
|
||||
- `mov rdest, [rdest + roff + variable_offset]` : access memory through normal GOAL deref.
|
||||
- Note - we should check that the register allocator gets this right always, and eliminates moves and avoid using a temporary register.
|
||||
- Again, the constant propagation should give use enough information, if we ever want/need to implement a more efficient `mov rdest, [rsp + variable_offset]` type instructions.
|
||||
- Memory access to static data should use `rip` addressing, like `mov rdest, [rip + offset]`. And creating pointers to static data could be `lea rdest, [rip - roff + offset]`
|
||||
@ -1,508 +0,0 @@
|
||||
# Compiler Example
|
||||
|
||||
This describes how the compiler works using the following code snippet, saved in a file named `example_goal.gc`.
|
||||
|
||||
```lisp
|
||||
(defun factorial-iterative ((x integer))
|
||||
(let ((result 1))
|
||||
(while (!= x 1)
|
||||
(set! result (* result x))
|
||||
(set! x (- x 1))
|
||||
)
|
||||
result
|
||||
)
|
||||
)
|
||||
|
||||
;; until we load KERNEL.CGO automatically, we have to do this to
|
||||
;; make format work correctly.
|
||||
(define-extern _format function)
|
||||
(define format _format)
|
||||
|
||||
(let ((x 10))
|
||||
(format #t "The value of ~D factorial is ~D~%" x (factorial-iterative x))
|
||||
)
|
||||
```
|
||||
|
||||
To run this yourself, start the compiler and runtime, then run:
|
||||
|
||||
```lisp
|
||||
(lt)
|
||||
(asm-file "doc/example_goal.gc" :color :load)
|
||||
```
|
||||
|
||||
And you should see:
|
||||
|
||||
```
|
||||
The value of 10 factorial is 3628800
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
The code to read from the GOAL REPL is in `Compiler.cpp`, in `Compiler::execute_repl`. Compiling an `asm-file` form will call `Compiler::compile_asm_file` in `CompilerControl.cpp`, which is where we'll start.
|
||||
|
||||
I've divided the process into these steps:
|
||||
1. __Read__: Convert from text to a representation of Lisp syntax.
|
||||
2. __IR-Pass__: Convert the S-Expressions to an intermediate representation (IR).
|
||||
3. __Register Allocation__: Map variables in the IR (`IRegister`s) to real hardware registers
|
||||
4. __Code Generation__: Generate x86-64 instructions from the IR
|
||||
5. __Object File Generation__: Put the instructions and static data in an object file and generate linking data
|
||||
6. __Sending__: Send the code to the runtime
|
||||
7. __Linking__: The runtime links the code so it can be run, and runs it.
|
||||
8. __Result__: The code prints the message which is sent back to the REPL.
|
||||
|
||||
## Reader
|
||||
|
||||
The reader converts GOAL/GOOS source into a `goos::Object`. One of the core ideas of lisp is that "code is data", so GOAL code is represented as GOOS data. This makes it easy for GOOS macros to operate on GOAL code. A GOOS object can represent a number, string, pair, etc. This strips out comments/whitespace.
|
||||
|
||||
The reader is run with this code:
|
||||
|
||||
```cpp
|
||||
auto code = m_goos.reader.read_from_file({filename});
|
||||
```
|
||||
|
||||
If you were to `code.print()`, you would get:
|
||||
|
||||
```lisp
|
||||
(top-level (defun factorial-iterative ((x integer)) (let ((result 1)) (while (!= x 1) (set! result (* result x)) (set! x (- x 1))) result)) (define-extern _format function) (define format _format) (let ((x 10)) (format #t "The value of ~D factorial is ~D~%" x (factorial-iterative x))))
|
||||
```
|
||||
|
||||
There are a few details worth mentioning about this process:
|
||||
- The reader will expand `'my-symbol` to `(quote my-symbol)`
|
||||
- The reader will throw errors on syntax errors (mismatched parentheses, bad strings/numbers, etc.)
|
||||
- Using `read_from_file` adds information about where each thing came from to a map stored in the reader. This map is used to determine the source file/line for compiler errors.
|
||||
|
||||
## IR Pass
|
||||
|
||||
This pass converts code (represented as a `goos::Object`) into intermediate representation. This is stored in an `Env*`, a tree structure. At the top is a `GlobalEnv*`, then an `FileEnv` for each file compiled, then a `FunctionEnv` for each function in the file. There are environments within `FunctionEnv` that are used for lexical scoping and the other types of GOAL scopes. The Intermediate Representation (IR) is a list per function that's built up in order as the compiler goes through the function. Note that the IR is a list of instructions and doesn't have a tree or more complicated structure. Here's an example of the IR for the example function:
|
||||
|
||||
```nasm
|
||||
Function: function-factorial-iterative
|
||||
mov-ic igpr-2, 1
|
||||
mov igpr-3, igpr-2
|
||||
goto label-10
|
||||
mov igpr-4, igpr-3
|
||||
imul igpr-4, igpr-0
|
||||
mov igpr-3, igpr-4
|
||||
mov igpr-5, igpr-0
|
||||
mov-ic igpr-6, 1
|
||||
subi igpr-5, igpr-6
|
||||
mov igpr-0, igpr-5
|
||||
mov-ic igpr-7, 1
|
||||
j(igpr-0 != igpr-7) label-3
|
||||
ret igpr-1 igpr-3
|
||||
|
||||
Function: function-top-level
|
||||
mov-fa igpr-0, function-factorial-iterative
|
||||
mov 'factorial-iterative, igpr-0
|
||||
mov igpr-1, '_format
|
||||
mov 'format, igpr-1
|
||||
mov-ic igpr-2, 10
|
||||
mov igpr-3, igpr-2
|
||||
mov igpr-4, '#t
|
||||
mov-sva igpr-5, static-string "The value of ~D factorial is ~D~%"
|
||||
mov igpr-6, 'factorial-iterative
|
||||
mov igpr-8, igpr-3
|
||||
call igpr-6 (ret igpr-7) (args igpr-8)
|
||||
mov igpr-9, igpr-7
|
||||
mov igpr-10, 'format
|
||||
mov igpr-12, igpr-4
|
||||
mov igpr-13, igpr-5
|
||||
mov igpr-14, igpr-3
|
||||
mov igpr-15, igpr-9
|
||||
call igpr-10 (ret igpr-11) (args igpr-12 igpr-13 igpr-14 igpr-15)
|
||||
mov igpr-16, igpr-11
|
||||
ret igpr-17 igpr-16
|
||||
```
|
||||
|
||||
The `function-top-level` is the "top level" function, which is everything not in a function. In the example, this is just defining the function, defining `format`, and calling `format`.
|
||||
|
||||
You'll notice that there are a ton of `mov`s between `igpr`. The compiler inserts tons of moves. Because this is all done in a single pass, there's a lot of cases where the compiler can't know if a move is needed or not. But the register allocator can figure it out and will remove most unneeded moves. Adding moves can also prevent stack spills. For example, consider the case where you want to get the return value of function `a`, then call function `b`, then call function `c` with the return value of function `a`. If there are a lot of moves, the register allocator can figure out a way to temporarily stash the value in a saved register instead of spilling to the stack.
|
||||
|
||||
Another thing to notice is that GOAL nested function calls suck. Example:
|
||||
|
||||
```lisp
|
||||
(format #t "The value of ~D factorial is ~D~%" x (factorial-iterative x))
|
||||
```
|
||||
|
||||
requires loading `format`, `#t`, the string, and `x` into registers, then calling `(factorial-iterative x)`, then calling `format`. This has to be done this way, just in case the `factorial-iterative` call modifies the value of `format`, `#t`, or `x`.
|
||||
|
||||
### IR Pass Implementation
|
||||
|
||||
An important type in the compiler is `Val`, which is a specification on how to get a value. A `Val` has an associated GOAL type (`TypeSpec`) and the IR Pass should take care of all type checking. A `Val` can represent a constant, a memory location (relative to pointer, or static data, etc), a spot in an array, a register etc. A `Val` representing a register is a `RegVal`, which contains an `IRegister`. An `IRegister` is a register that's not yet mapped to the hardware, and instead has a unique integer to identify itself. The IR assumes there are infinitely many `IRegister`s, and a later stage maps `IRegister`s to real hardware registers.
|
||||
|
||||
The general process starts with a `compile` function, which dispatches other `compile_<thing>` functions as needed. These generally take in `goos::Object` as a code input, emit IR into an `Env`/or modify things in the `Env`, and return a `Val*` describing the result.
|
||||
|
||||
In general, GOAL is very greedy and `compile` functions emit IR to do things, then put the result in a register, and return a `RegVal`.
|
||||
|
||||
However, there is an exception for memory related things. Consider
|
||||
|
||||
```lisp
|
||||
(-> my-object my-field) ; my_object->my_field in C
|
||||
```
|
||||
|
||||
This shouldn't return a `Val` for a register containing the value of `my_object->my_field`, but should instead return something that represents "the memory location of my_field in my_object". This way you can do
|
||||
|
||||
```lisp
|
||||
(set! (-> my-object my-field) val)
|
||||
;; or
|
||||
(& (-> my-object my-field)) ;; &my_object->my_field
|
||||
```
|
||||
|
||||
and the compiler will have enough information to figure out the memory address.
|
||||
|
||||
If the compiler actually needs the value of something, and wants to be sure its a value in a register, it will use the `to_reg` method of `Val`. This will emit IR into the current function to get the value in a register, then return a `Val` that represents this register. Example `to_reg` implementation for integer constants:
|
||||
|
||||
```cpp
|
||||
RegVal* IntegerConstantVal::to_reg(Env* fe) {
|
||||
auto rv = fe->make_gpr(m_ts);
|
||||
fe->emit(std::make_unique<IR_LoadConstant64>(rv, m_value));
|
||||
return rv;
|
||||
}
|
||||
```
|
||||
|
||||
Note that `to_reg` can emit code like reading from memory, where the order of operations really matters, so you have to be very careful.
|
||||
|
||||
It's extremely dangerous to let a memory reference `Val` propagate too far. Consider this example:
|
||||
|
||||
```lisp
|
||||
(let ((x (-> my-object my-field)))
|
||||
(set! (-> my-object my-field) 12)
|
||||
x
|
||||
)
|
||||
```
|
||||
|
||||
Where `x` should be the old value of `my-field`. The `Val` for `x` needs to be `to_reg`ed _before_ getting inside the `let`. There's also some potential confusion around the order that you compile and `to_gpr` things. In a case where you need a bunch of values in gprs, you should do the `to_gpr` immediately after compiling to match the exact behavior of the original GOAL. For example
|
||||
|
||||
```lisp
|
||||
(+ (-> my-array (* x y)) (some-function))
|
||||
;; like c++ my_array[x*y] + some_function()
|
||||
```
|
||||
|
||||
When we `compile` the `(-> my-array (* x y))`, it will emit code to calculate the `(*x y)`, but won't actually do the memory access until we call `to_reg` on the result. This memory access should happen __before__ `some-function` is called.
|
||||
|
||||
In general, each time you `compile` something, you should immediately `to_gpr` it, _before_ `compile`ing the next thing. Many places will only accept a `RegVal` as an input to help with this. Also, the result for almost all compilation functions should be `to_reg`ed. The only exceptions are forms which deal with memory references (address of operator, dereference operator) or math.
|
||||
|
||||
Another important thing is that compilation functions should _never_ modify any existing `IRegister`s or `Val`s, unless that function is `set!`, which handles the situation correctly. Instead, create new `IRegister`s and move into those. I am planning to implement a `settable` flag to help reduce errors.
|
||||
|
||||
For example:
|
||||
- `RegVal` storing a local variable: is `settable`, you can modify local variables by writing to the register they use.
|
||||
- `RegVal` storing the result of a memory dereference: not `settable`, you should set the memory instead.
|
||||
- `RegVal` containing the result of converting a `xmm` to `gpr`: not settable, you need to set the original `xmm` instead
|
||||
|
||||
The only settable `RegVal` is one corresponding to a local variable.
|
||||
|
||||
### Following the Code
|
||||
|
||||
This pass runs from here:
|
||||
|
||||
```cpp
|
||||
auto obj_file = compile_object_file(obj_file_name, code, !no_code);
|
||||
```
|
||||
|
||||
That function sets up a `FileEnv*`, then runs
|
||||
|
||||
```cpp
|
||||
file_env->add_top_level_function(
|
||||
compile_top_level_function("top-level", std::move(code), compilation_env));
|
||||
```
|
||||
|
||||
which compiles the body of the function with:
|
||||
|
||||
```cpp
|
||||
auto result = compile_error_guard(code, fe.get());
|
||||
```
|
||||
|
||||
The `compile_error_guard` function takes in code (as a `goos::Object`) and a `Env*`, and returns a `Val` representing the return value of the code. It calls the `compile` function, but wraps it in a `try catch` block to catch any compilation errors and print an error message. In the case where there's no error, it just does:
|
||||
|
||||
```cpp
|
||||
return compile(code, env);
|
||||
```
|
||||
|
||||
The `compile` function is pretty simple:
|
||||
|
||||
```cpp
|
||||
/*!
|
||||
* Highest level compile function
|
||||
*/
|
||||
Val* Compiler::compile(const goos::Object& code, Env* env) {
|
||||
switch (code.type) {
|
||||
case goos::ObjectType::PAIR:
|
||||
return compile_pair(code, env);
|
||||
case goos::ObjectType::INTEGER:
|
||||
return compile_integer(code, env);
|
||||
case goos::ObjectType::SYMBOL:
|
||||
return compile_symbol(code, env);
|
||||
case goos::ObjectType::STRING:
|
||||
return compile_string(code, env);
|
||||
case goos::ObjectType::FLOAT:
|
||||
return compile_float(code, env);
|
||||
default:
|
||||
ice("Don't know how to compile " + code.print());
|
||||
}
|
||||
return get_none();
|
||||
}
|
||||
```
|
||||
|
||||
In our case, the code starts with `(defun..`, which is actually a GOOS macro. It throws away the docstring, creates a lambda, then stores the function in a symbol:
|
||||
|
||||
```lisp
|
||||
;; Define a new function
|
||||
(defmacro defun (name bindings &rest body)
|
||||
(if (and
|
||||
(> (length body) 1) ;; more than one thing in function
|
||||
(string? (first body)) ;; first thing is a string
|
||||
)
|
||||
;; then it's a docstring and we ignore it.
|
||||
`(define ,name (lambda :name ,name ,bindings ,@(cdr body)))
|
||||
;; otherwise don't ignore it.
|
||||
`(define ,name (lambda :name ,name ,bindings ,@body))
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
The compiler notices this is a macro in `compile_pair`:
|
||||
|
||||
```cpp
|
||||
if (head.is_symbol()) {
|
||||
// ...
|
||||
|
||||
goos::Object macro_obj;
|
||||
if (try_getting_macro_from_goos(head, ¯o_obj)) {
|
||||
return compile_goos_macro(code, macro_obj, rest, env);
|
||||
}
|
||||
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
The `compile_goos_macro` function sets up a GOOS environment and interprets the GOOS macro to generate more GOAL code:
|
||||
|
||||
```cpp
|
||||
Val* Compiler::compile_goos_macro(const goos::Object& o,
|
||||
const goos::Object& macro_obj,
|
||||
const goos::Object& rest,
|
||||
Env* env) {
|
||||
auto macro = macro_obj.as_macro();
|
||||
Arguments args = m_goos.get_args(o, rest, macro->args);
|
||||
auto mac_env_obj = EnvironmentObject::make_new(); // GOOS environment
|
||||
auto mac_env = mac_env_obj.as_env();
|
||||
mac_env->parent_env = m_goos.global_environment.as_env();
|
||||
m_goos.set_args_in_env(o, args, macro->args, mac_env);
|
||||
auto goos_result = m_goos.eval_list_return_last(macro->body, macro->body, mac_env); // evaluate GOOS macro
|
||||
return compile_error_guard(goos_result, env); // compile resulting GOAL code
|
||||
}
|
||||
```
|
||||
|
||||
and the last line of that function compiles the result of macro expansion in GOAL.
|
||||
|
||||
As an example, I'm going to look at `compile_add`, which handles the `+` form, and is representative of typical compiler code for this part. We start by checking that the arguments look valid:
|
||||
|
||||
```cpp
|
||||
Val* Compiler::compile_add(const goos::Object& form, const goos::Object& rest, Env* env) {
|
||||
auto args = get_va(form, rest); // get arguments to + in a list
|
||||
if (!args.named.empty() || args.unnamed.empty()) {
|
||||
throw_compile_error(form, "Invalid + form");
|
||||
}
|
||||
```
|
||||
|
||||
Then we compile the first thing in the `(+ ...` form, get its type, and pick a math mode (int, float):
|
||||
|
||||
```cpp
|
||||
auto first_val = compile_error_guard(args.unnamed.at(0), env);
|
||||
auto first_type = first_val->type();
|
||||
auto math_type = get_math_mode(first_type);
|
||||
```
|
||||
|
||||
In the integer case, we first create a new variable in the IR called an `IRegister` that must be in a GPR (as opposed to an XMM floating point register), and then emit an IR instruction that sets this result register to the first argument.
|
||||
|
||||
```cpp
|
||||
auto result = env->make_gpr(first_type);
|
||||
env->emit(std::make_unique<IR_RegSet>(result, first_val->to_gpr(env)));
|
||||
```
|
||||
|
||||
Then, for each of the remaining arguments, we do:
|
||||
|
||||
```cpp
|
||||
for (size_t i = 1; i < args.unnamed.size(); i++) {
|
||||
env->emit(std::make_unique<IR_IntegerMath>(
|
||||
IntegerMathKind::ADD_64, result,
|
||||
to_math_type(compile_error_guard(args.unnamed.at(i), env), math_type, env)
|
||||
->to_gpr(env)));
|
||||
}
|
||||
```
|
||||
|
||||
which emits an IR to add the value to the sum. The `to_math_type` will emit any code needed to convert this to the correct numeric type (returns either a numeric constant or a `RegVal` containing the value).
|
||||
|
||||
An important detail is that we create a new register which will hold the result. This may seem inefficient in cases, but a later compile pass will try to make this new register be the same register as `first_val` if possible, and will eliminate the `IR_RegSet`.
|
||||
|
||||
## Register Allocation
|
||||
|
||||
This step figures out how to match up `IRegister`s to real hardware registers. In the case where there aren't enough hardware registers, it figures out how to "spill" variables onto the stack. The current implementation is a very greedy one, so it doesn't always succeed at doing things perfectly. The stack spilling is also not handled very efficiently, but the hope is that most functions won't require stack spilling.
|
||||
|
||||
This step is run from `compile_asm_function` on the line:
|
||||
|
||||
```cpp
|
||||
color_object_file(obj_file);
|
||||
|
||||
void Compiler::color_object_file(FileEnv* env) {
|
||||
for (auto& f : env->functions()) {
|
||||
AllocationInput input;
|
||||
for (auto& i : f->code()) {
|
||||
input.instructions.push_back(i->to_rai());
|
||||
input.debug_instruction_names.push_back(i->print());
|
||||
}
|
||||
input.max_vars = f->max_vars();
|
||||
input.constraints = f->constraints();
|
||||
|
||||
// debug prints removed
|
||||
|
||||
f->set_allocations(allocate_registers(input));
|
||||
}
|
||||
}
|
||||
```
|
||||
The actual algorithm is too complicated to describe here, but it figures out a mapping from `IRegister`s to hardware registers. It also figures out how much space on the stack is needed for any stack spills, which saved registers will be used, and deals with aligning the stack.
|
||||
|
||||
## Code Generation
|
||||
This part actually generates the static data and x86 instructions and stores them in an `ObjectGenerator`. See `CodeGenerator::do_function`. It emits the function prologue and epilogue, as well as any extra loads/stores from the stack that the register allocator added. Each `IR` gets to emit instructions with:
|
||||
|
||||
```cpp
|
||||
ir->do_codegen(&m_gen, allocs, i_rec);
|
||||
```
|
||||
|
||||
Each IR has its own `do_codegen` that emits the right instruction, and also any linking data that's needed. For example, instructions that access the symbol table are patched by the runtime to directly access the correct slot of the hash table, so the `do_codegen` also lets the `ObjectGenerator` know about this link:
|
||||
|
||||
```cpp
|
||||
// IR_GetSymbolValue::do_codegen
|
||||
|
||||
// look at register allocation result to determine hw register
|
||||
auto dst_reg = get_reg(m_dest, allocs, irec);
|
||||
|
||||
// add an instruction
|
||||
auto instr = gen->add_instr(IGen::load32u_gpr64_gpr64_plus_gpr64_plus_s32(
|
||||
dst_reg, gRegInfo.get_st_reg(), gRegInfo.get_offset_reg(), 0x0badbeef), irec);
|
||||
|
||||
// add link info
|
||||
gen->link_instruction_symbol_mem(instr, m_src->name());
|
||||
```
|
||||
|
||||
here `0xbadbeef` is used as a placeholder offset - the runtime should patch this to the actual offset of the symbol.
|
||||
|
||||
There's a ton of book-keeping to figure out the correct offsets for `rip`-relative addressing, or how to deal with jumps to/from IR which become multiple (or zero!) x86-64 instructions. It should all be handled by `ObjectFileGenerator`, and not in `do_codegen` or `CodeGenerator`.
|
||||
|
||||
## Object File Generation
|
||||
Once the `CodeGenerator` is done going through all functions and static data, it runs:
|
||||
|
||||
```cpp
|
||||
return m_gen.generate_data_v3().to_vector();
|
||||
```
|
||||
|
||||
This actually lays out everything in memory. It takes a few passes because x86 instructions are variable length (may even change based on which registers are used!), so it's a little bit tricky to figure out offsets between different instructions or instructions and data. Finally it generates link data tables, which efficiently pack together links to the same symbols into a single entry, to avoid duplicated symbol names. The link table also contains information about linking references in between different segments, as different parts of the object file may be loaded into different spots in memory, and will need to reference each other.
|
||||
|
||||
This is the final result for top-level function (stored in top-level segment)
|
||||
|
||||
```nasm
|
||||
;; prologue
|
||||
push rbx
|
||||
push rbp
|
||||
push r10
|
||||
push r11
|
||||
push rbx
|
||||
|
||||
;; load address of factorial-iterative function
|
||||
lea rax,[rip+0x0]
|
||||
;; convert to GOAL pointer
|
||||
sub rax,r15
|
||||
;; store in symbol table
|
||||
mov DWORD PTR [r15+r14*1+0xbadbeef],eax
|
||||
|
||||
;; load _format value
|
||||
mov eax,DWORD PTR [r15+r14*1+0xbadbeef]
|
||||
;; store in format
|
||||
mov DWORD PTR [r15+r14*1+0xbadbeef],eax
|
||||
|
||||
;; load constant 10 (the greedy regalloc does poorly here)
|
||||
mov eax,0xa
|
||||
mov rbx,rax
|
||||
|
||||
;; load format from symbol table
|
||||
mov ebp,DWORD PTR [r15+r14*1+0xbadbeef]
|
||||
|
||||
;; load #t from symbol table
|
||||
mov r10d,DWORD PTR [r15+r14*1+0xbadbeef]
|
||||
|
||||
;; get address of string
|
||||
lea r11,[rip+0x0]
|
||||
;; convert to GOAL pointer
|
||||
sub r11,r15
|
||||
|
||||
;; get factorial-iterative from symbol table
|
||||
mov ecx,DWORD PTR [r15+r14*1+0xbadbeef]
|
||||
;; move 10 into argument register
|
||||
mov rdi,rbx
|
||||
;; convert factorial-iterative to a real pointer
|
||||
add rcx,r15
|
||||
;; call factorial
|
||||
call rcx
|
||||
|
||||
;; move args into argument registers
|
||||
mov rdi,r10
|
||||
mov rsi,r11
|
||||
mov rdx,rbx
|
||||
mov rcx,rax
|
||||
|
||||
;; call format
|
||||
add rbp,r15
|
||||
call rbp
|
||||
|
||||
;; epilogue
|
||||
pop rbx
|
||||
pop r11
|
||||
pop r10
|
||||
pop rbp
|
||||
pop rbx
|
||||
ret
|
||||
```
|
||||
|
||||
and the factorial function (stored in main segment)
|
||||
|
||||
```nasm
|
||||
mov eax,0x1
|
||||
jmp 0x18
|
||||
mul eax,edi
|
||||
movsxd rax,eax
|
||||
mov ecx,0x1
|
||||
sub rdi,rcx
|
||||
mov ecx,0x1
|
||||
cmp rdi,rcx
|
||||
jne 0xa
|
||||
ret
|
||||
```
|
||||
|
||||
## Sending and Receiving
|
||||
|
||||
The result of `codegen_object_file` is sent with:
|
||||
|
||||
```cpp
|
||||
m_listener.send_code(data);
|
||||
```
|
||||
|
||||
which adds a message header then just sends the code over the socket.
|
||||
|
||||
The receive process is complicated, so this is just a quick summary
|
||||
- `Deci2Server` receives it
|
||||
- calls `GoalProtoHandler` in chunks, which stores it in a buffer (`MessBuffArea`)
|
||||
- Once fully received, `WaitForMessageAndAck` will return a pointer to the message. The name of this function is totally wrong, it doesn't wait for a message, and it doesn't ack the message.
|
||||
- The main loop in `KernelCheckAndDispatch` will see this message and run `ProcessListenerMessage`
|
||||
- `ProcessListenerMessage` sees that it has code, copies the message to the debug heap, and links it.
|
||||
|
||||
```cpp
|
||||
auto buffer = kmalloc(kdebugheap, MessCount, 0, "listener-link-block");
|
||||
memcpy(buffer.c(), msg.c(), MessCount);
|
||||
ListenerLinkBlock->value = buffer.offset + 4;
|
||||
ListenerFunction->value = link_and_exec(buffer, "*listener*", 0, kdebugheap, LINK_FLAG_FORCE_DEBUG).offset;
|
||||
```
|
||||
|
||||
- The `link_and_exec` function doesn't actually execute anything because it doesn't have the `LINK_FLAG_EXECUTE` set, it just links things. It moves the top level function and linking data to the top of the heap (temporary storage for the kernel) and keep both the main segment and debug segment of the code on the debug heap. It'll move them together and eliminate gaps before linking. After linking, the `ListenerFunction->value` will contain a pointer to the top level function, which is stored in the top temp area of the heap. This `ListenerFunction` is the GOAL `*listener-function*` symbol.
|
||||
- The next time the GOAL kernel runs, it will notice that `*listener-function*` is set, then call this function, then set it to `#f` to indicate it called the function.
|
||||
- After this, `ClearPending()` is called, which sends all of the `print` messages with the `Deci2Server` back to the compiler.
|
||||
- Because the GOAL kernel changed `ListenerFunction` to `#f`, it does a `SendAck()` to send a special `ACK` message to the compiler, saying "I got the function, ran it, and didn't crash. Now I'm ready for more messages."
|
||||
@ -1,431 +0,0 @@
|
||||
# OpenGOAL Debugger
|
||||
|
||||
The debugger works on Windows and Linux. All the platform specific code is in `xdbg.cpp`. When attached to a target, things like exceptions from invalid memory access or divides by zero break into the debugger for inspection on the code or values that caused the break. The technical implementation of the debugger across multiple platforms means there will be a few differences in how it handles or displays certain things. For example, the debugger on Linux will break if the GOAL (EE) thread runs into a breakpoint, but on Windows this can be caused by any thread on the target as the thread that `(:break)` breaks is unspecified.
|
||||
|
||||
## Commands
|
||||
|
||||
### `(dbs)`
|
||||
|
||||
Print the status of the debugger and listener. The listener status is whether or not there is a socket connection open between the compiler and the target. The "debug context" is information that the runtime sends to the compiler so it can find the correct thread to debug. In order to debug, you need both.
|
||||
|
||||
### `(dbg)`
|
||||
|
||||
Attach the debugger. This will stop the target.
|
||||
|
||||
Example of connecting to the target for debugging:
|
||||
|
||||
```lisp
|
||||
OpenGOAL Compiler 0.1
|
||||
|
||||
;; attach the listener over the network
|
||||
g> (lt)
|
||||
[Listener] Socket connected established! (took 0 tries). Waiting for version...
|
||||
Got version 0.1 OK!
|
||||
|
||||
;; this message is sent from the target from kprint.cpp and contains the "debug context"
|
||||
[OUTPUT] reset #x147d24 #x2000000000 1062568
|
||||
|
||||
;; the debugger gets the message and remembers it so it can connect in the future.
|
||||
[Debugger] Context: valid = true, s7 = 0x147d24, base = 0x2000000000, tid = 1062568
|
||||
|
||||
;; attach the debugger and halt
|
||||
gc> (dbg)
|
||||
[Debugger] PTRACE_ATTACHED! Waiting for process to stop...
|
||||
Debugger connected.
|
||||
|
||||
;; print the debugger status
|
||||
gc> (dbs)
|
||||
Listener connected? true
|
||||
Debugger context? true
|
||||
Attached? true
|
||||
Halted? true
|
||||
Context: valid = true, s7 = 0x147d24, base = 0x2000000000, tid = 1062568
|
||||
```
|
||||
|
||||
### `(:cont)`
|
||||
|
||||
Continue the target if it has been stopped.
|
||||
|
||||
### `(:stop)`
|
||||
|
||||
Detach from target.
|
||||
|
||||
### `(:break)`
|
||||
|
||||
Immediately stop the target if it is running. Will print some registers.
|
||||
|
||||
### `(:dump-all-mem <path>)`
|
||||
|
||||
Dump all GOAL memory to a file. Must be stopped.
|
||||
|
||||
```lisp
|
||||
(:dump-all-mem "mem.bin")
|
||||
```
|
||||
|
||||
The path is relative to the Jak project folder.
|
||||
|
||||
The file will be the exact size of `EE_MAIN_MEM_SIZE`, but the first `EE_LOW_MEM_PROTECT` bytes are zero, as these cannot be written or read.
|
||||
|
||||
## Address Spec
|
||||
|
||||
Anywhere an address can be used, you can also use an "address spec", which gives you easier ways to input addresses. For now, the address spec is pretty simple, but there will be more features in the future.
|
||||
|
||||
- `(sym-val <sym-name>)`. Get the address stored in the symbol with the given name. Currently there's no check to see if the symbol actually stores an address or not. This is like "evaluate `<sym-name>`, then treat the value as an address"
|
||||
- `(sym <sym-name>)`. Get the address of the symbol object itself, including the basic offset.
|
||||
|
||||
Example to show the difference:
|
||||
|
||||
```lisp
|
||||
|
||||
;; the symbol is at 0x142d1c
|
||||
gc> (inspect '*kernel-context*)
|
||||
[ 142d1c] symbol
|
||||
name: *kernel-context*
|
||||
hash: #x8f9a35ff
|
||||
value: #<kernel-context @ #x164a84>
|
||||
1322268
|
||||
|
||||
;; the object is at 0x164a84
|
||||
gc> (inspect *kernel-context*)
|
||||
[00164a84] kernel-context
|
||||
prevent-from-run: 65
|
||||
require-for-run: 0
|
||||
allow-to-run: 0
|
||||
next-pid: 2
|
||||
fast-stack-top: 1879064576
|
||||
current-process: #f
|
||||
relocating-process: #f
|
||||
relocating-min: 0
|
||||
relocating-max: 0
|
||||
relocating-offset: 0
|
||||
low-memory-message: #t
|
||||
1460868
|
||||
|
||||
;; break, so we can debug
|
||||
gc> (:break)
|
||||
Read symbol table (159872 bytes, 226 reads, 225 symbols, 1.96 ms)
|
||||
rax: 0xfffffffffffffdfc rcx: 0x00007f745b508361 rdx: 0x00007f745b3ffca0 rbx: 0x0000000000147d24
|
||||
rsp: 0x00007f745b3ffc40 rbp: 0x00007f745b3ffcc0 rsi: 0x0000000000000000 rdi: 0x0000000000000000
|
||||
r8: 0x0000000000000000 r9: 0x0000000000000008 r10: 0x00007f745b3ffca0 r11: 0x0000000000000293
|
||||
r12: 0x0000000000147d24 r13: 0x00007ffdff32cfaf r14: 0x00007ffdff32cfb0 r15: 0x00007f745b3fffc0
|
||||
rip: 0x00007f745b508361
|
||||
|
||||
;; reads the symbol's memory:
|
||||
;; at 0x142d1c there is the value 0x164a84
|
||||
gc> (dw (sym *kernel-context*) 1)
|
||||
0x00142d1c: 0x00164a84
|
||||
|
||||
;; treat the symbol's value as an address and read the memory there.
|
||||
;; notice that the 0x41 in the first word is decimal 65, the first field of the kernel-context.
|
||||
gc> (dw (sym-val *kernel-context*) 10)
|
||||
0x00164a84: 0x00000041 0x00000000 0x00000000 0x00000002
|
||||
0x00164a94: 0x70004000 0x00147d24 0x00147d24 0x00000000
|
||||
0x00164aa4: 0x00000000 0x00000000
|
||||
```
|
||||
|
||||
|
||||
### `(:pm)`
|
||||
|
||||
Print memory
|
||||
|
||||
```lisp
|
||||
(:pm elt-size addr elt-count [:print-mode mode])
|
||||
```
|
||||
|
||||
The element size is the size of each word to print. It can be 1, 2, 4, 8 currently. The address is the GOAL Address to print at. The elt-count is the number of words to print. The print mode is option and defaults to `hex`. There is also an `unsigned-decimal`, a `signed-decimal`, and `float`. The `float` mode only works when `elt-size` is 4.
|
||||
|
||||
There are some useful macros inspired by the original PS2 TOOL debugger (`dsedb`) for the different sizes. They are `db`, `dh`, `dw`, and `dd` for 1, 2, 4, and 8 byte hex prints which follows the naming convention of MIPS load/stores. There is also a `df` for printing floats. See the example below.
|
||||
|
||||
```lisp
|
||||
OpenGOAL Compiler 0.1
|
||||
|
||||
;; first connect the listener
|
||||
g> (lt)
|
||||
[Listener] Socket connected established! (took 0 tries). Waiting for version...
|
||||
Got version 0.1 OK!
|
||||
[OUTPUT] reset #x147d24 #x2000000000 53371
|
||||
|
||||
[Debugger] Context: valid = true, s7 = 0x147d24, base = 0x2000000000, tid = 53371
|
||||
|
||||
;; define a new array of floats, and set a few values
|
||||
gc> (define x (new 'global 'array 'float 12))
|
||||
1452224
|
||||
|
||||
gc> (set! (-> x 0) 1.0)
|
||||
1065353216
|
||||
|
||||
gc> (set! (-> x 2) 2.0)
|
||||
1073741824
|
||||
|
||||
;; attach the debugger (halts the target)
|
||||
gc> (dbg)
|
||||
[Debugger] PTRACE_ATTACHED! Waiting for process to stop...
|
||||
rax: 0xfffffffffffffdfc rcx: 0x00007f6b94964361 rdx: 0x00007f6b8fffeca0 rbx: 0x0000000000147d24
|
||||
rsp: 0x00007f6b8fffec40 rbp: 0x00007f6b8fffecc0 rsi: 0x0000000000000000 rdi: 0x0000000000000000
|
||||
r8: 0x0000000000000000 r9: 0x000000000000000b r10: 0x00007f6b8fffeca0 r11: 0x0000000000000293
|
||||
r12: 0x0000000000147d24 r13: 0x00007ffd16fb117f r14: 0x00007ffd16fb1180 r15: 0x00007f6b8fffefc0
|
||||
rip: 0x00007f6b94964361
|
||||
Debugger connected.
|
||||
|
||||
;; print memory as 10 bytes
|
||||
gc> (db 1452224 10)
|
||||
0x001628c0: 0x00 0x00 0x80 0x3f 0x00 0x00 0x00 0x00 0x00 0x00
|
||||
|
||||
;; print memory as 10 words (32-bit words)
|
||||
gc> (dw 1452224 10)
|
||||
0x001628c0: 0x3f800000 0x00000000 0x40000000 0x00000000
|
||||
0x001628d0: 0x00000000 0x00000000 0x00000000 0x00000000
|
||||
0x001628e0: 0x00000000 0x00000000
|
||||
|
||||
;; print memory as 10 floats
|
||||
gc> (df 1452224 10)
|
||||
0x001628c0: 1.0000 0.0000 2.0000 0.0000
|
||||
0x001628d0: 0.0000 0.0000 0.0000 0.0000
|
||||
0x001628e0: 0.0000 0.0000
|
||||
|
||||
;; set some more values, must unbreak first
|
||||
gc> (:cont)
|
||||
gc> (set! (-> x 1) (the-as float -12))
|
||||
-12
|
||||
|
||||
;; break and print as decimal
|
||||
gc> (:break)
|
||||
rax: 0xfffffffffffffdfc rcx: 0x00007f6b94964361 rdx: 0x00007f6b8fffeca0 rbx: 0x0000000000147d24
|
||||
rsp: 0x00007f6b8fffec40 rbp: 0x00007f6b8fffecc0 rsi: 0x0000000000000000 rdi: 0x0000000000000000
|
||||
r8: 0x0000000000000000 r9: 0x0000000000000004 r10: 0x00007f6b8fffeca0 r11: 0x0000000000000293
|
||||
r12: 0x0000000000147d24 r13: 0x00007ffd16fb117f r14: 0x00007ffd16fb1180 r15: 0x00007f6b8fffefc0
|
||||
rip: 0x00007f6b94964361
|
||||
gc> (:pm 4 1452224 10 :print-mode unsigned-dec)
|
||||
0x001628c0: 1065353216 4294967284 1073741824 0
|
||||
0x001628d0: 0 0 0 0
|
||||
0x001628e0: 0 0
|
||||
gc> (:pm 4 1452224 10 :print-mode signed-dec)
|
||||
0x001628c0: 1065353216 -12 1073741824 0
|
||||
0x001628d0: 0 0 0 0
|
||||
0x001628e0: 0 0
|
||||
```
|
||||
|
||||
### `(:disasm)`
|
||||
|
||||
Disassembly instructions in memory
|
||||
|
||||
```lisp
|
||||
(:disasm addr len)
|
||||
```
|
||||
|
||||
Example (after doing a `(lt)`, `(blg)`, `(dbg)`):
|
||||
|
||||
```nasm
|
||||
gs> (:disasm (sym-val basic-type?) 59)
|
||||
Object: gcommon
|
||||
|
||||
[0x28eb4c631c4] mov r9d, [r15+rdi*1-0x04]
|
||||
[0x28eb4c631c9] mov r8d, [object]
|
||||
[0x28eb4c631d1] cmp r9, rsi
|
||||
[0x28eb4c631d4] jnz 0x0000028EB4C631E9
|
||||
[0x28eb4c631da] lea rax, '#t
|
||||
[0x28eb4c631df] jmp 0x0000028EB4C631FD
|
||||
[0x28eb4c631e4] jmp 0x0000028EB4C631EC
|
||||
[0x28eb4c631e9] mov rcx, '#f
|
||||
[0x28eb4c631ec] mov r9d, [r15+r9*1+0x04]
|
||||
[0x28eb4c631f1] cmp r9, r8
|
||||
[0x28eb4c631f4] jnz 0x0000028EB4C631D1
|
||||
[0x28eb4c631fa] mov rax, '#f
|
||||
[0x28eb4c631fd] ret
|
||||
```
|
||||
|
||||
For now, the disassembly is pretty basic, but it should eventually support GOAL symbols.
|
||||
|
||||
### `(:sym-name)`
|
||||
|
||||
Print the name of a symbol from its offset. The name is fetched from memory.
|
||||
|
||||
```lisp
|
||||
(:sym-name offset)
|
||||
```
|
||||
|
||||
Example (after doing a `(lt)`, `(blg)`, `(dbg)`):
|
||||
|
||||
```nasm
|
||||
gs> (:sym-name 0)
|
||||
symbol name for symbol 0h is #f
|
||||
gs> (:sym-name 8)
|
||||
symbol name for symbol 8h is #t
|
||||
gs> (:sym-name 16)
|
||||
symbol name for symbol 10h is function
|
||||
gs> (:sym-name #x18)
|
||||
symbol name for symbol 18h is basic
|
||||
gs> (:sym-name #x20)
|
||||
symbol name for symbol 20h is string
|
||||
gs> (:sym-name #x30)
|
||||
symbol name for symbol 30h is type
|
||||
gs> (:sym-name #x80)
|
||||
symbol name for symbol 80h is int64
|
||||
gs> (:sym-name #x800)
|
||||
symbol 800h is not loaded or is invalid
|
||||
```
|
||||
|
||||
Keep in mind `-#xa8` is not valid syntax for a negative number in hexadecimal.
|
||||
|
||||
## Breakpoints
|
||||
|
||||
```lisp
|
||||
OpenGOAL Compiler 0.1
|
||||
|
||||
;; first, connect to the target
|
||||
g > (lt)
|
||||
[Listener] Socket connected established! (took 0 tries). Waiting for version...
|
||||
Got version 0.1 OK!
|
||||
[OUTPUT] reset #x147d24 #x2000000000 322300
|
||||
|
||||
[Debugger] Context: valid = true, s7 = 0x147d24, base = 0x2000000000, tid = 322300
|
||||
|
||||
|
||||
;; run an infinite loop. This will time out because we don't see a response from the GOAL kernel that our function
|
||||
;; has returned.
|
||||
gc > (while #t (+ 1 2 3 4 5 6 7))
|
||||
Error - target has timed out. If it is stuck in a loop, it must be manually killed.
|
||||
Runtime is not responding. Did it crash?
|
||||
|
||||
|
||||
;; so we can attach the debugger!
|
||||
gc > (dbg)
|
||||
[Debugger] PTRACE_ATTACHED! Waiting for process to stop...
|
||||
Target has stopped. Run (:di) to get more information.
|
||||
Read symbol table (146816 bytes, 124 reads, 123 symbols, 2.02 ms)
|
||||
rax: 0x000000000000000a rcx: 0x0000000000000005 rdx: 0x0000000000000000 rbx: 0x0000002000000000
|
||||
rsp: 0x00007fddcde75c58 rbp: 0x00007fddcde75cc0 rsi: 0x0000000000000000 rdi: 0x0000000000000000
|
||||
r8: 0x0000000000147d24 r9: 0x0000002000000000 r10: 0x00007fddcde75ca0 r11: 0x0000000000000000
|
||||
r12: 0x0000000000147d24 r13: 0x0000002007ffbf14 r14: 0x0000000000147d24 r15: 0x0000002000000000
|
||||
rip: 0x0000002007ffbf3b
|
||||
[0x2007ffbf1b] add [rax], al
|
||||
[0x2007ffbf1d] add [rcx+0x02], bh
|
||||
[0x2007ffbf23] add rax, rcx
|
||||
[0x2007ffbf26] mov ecx, 0x03
|
||||
[0x2007ffbf2b] add rax, rcx
|
||||
[0x2007ffbf2e] mov ecx, 0x04
|
||||
[0x2007ffbf33] add rax, rcx
|
||||
[0x2007ffbf36] mov ecx, 0x05
|
||||
- [0x2007ffbf3b] add rax, rcx
|
||||
[0x2007ffbf3e] mov ecx, 0x06
|
||||
[0x2007ffbf43] add rax, rcx
|
||||
[0x2007ffbf46] mov ecx, 0x07
|
||||
[0x2007ffbf4b] add rax, rcx
|
||||
[0x2007ffbf4e] mov eax, [r15+r14*1+0x08]
|
||||
[0x2007ffbf56] mov rcx, r14
|
||||
[0x2007ffbf59] add rcx, 0x00
|
||||
[0x2007ffbf60] cmp rax, rcx
|
||||
[0x2007ffbf63] jnz 0x0000002007FFBF19
|
||||
[0x2007ffbf69] mov eax, [r15+r14*1]
|
||||
[0x2007ffbf71] ret
|
||||
[0x2007ffbf72] add [rax], al
|
||||
[0x2007ffbf74] add [rax], al
|
||||
[0x2007ffbf76] add [rax], al
|
||||
[0x2007ffbf78] add [rax], al
|
||||
[0x2007ffbf7a] INVALID (0x00)
|
||||
|
||||
Debugger connected.
|
||||
|
||||
;; currently rcx = 5. let's set a breakpoint where it should be 7
|
||||
gcs> (:bp #x2007ffbf4b)
|
||||
|
||||
;; and continue...
|
||||
gcs> (:cont)
|
||||
|
||||
;; it hits the breakpoint. (this message should have more information...)
|
||||
Target has stopped. Run (:di) to get more information.
|
||||
|
||||
;; get some info:
|
||||
gcs> (:di)
|
||||
Read symbol table (146816 bytes, 124 reads, 123 symbols, 1.46 ms)
|
||||
rax: 0x0000000000000015 rcx: 0x0000000000000007 rdx: 0x0000000000000000 rbx: 0x0000002000000000
|
||||
rsp: 0x00007fddcde75c58 rbp: 0x00007fddcde75cc0 rsi: 0x0000000000000000 rdi: 0x0000000000000000
|
||||
r8: 0x0000000000147d24 r9: 0x0000002000000000 r10: 0x00007fddcde75ca0 r11: 0x0000000000000000
|
||||
r12: 0x0000000000147d24 r13: 0x0000002007ffbf14 r14: 0x0000000000147d24 r15: 0x0000002000000000
|
||||
rip: 0x0000002007ffbf4c
|
||||
[0x2007ffbf2c] add eax, ecx
|
||||
[0x2007ffbf2e] mov ecx, 0x04
|
||||
[0x2007ffbf33] add rax, rcx
|
||||
[0x2007ffbf36] mov ecx, 0x05
|
||||
[0x2007ffbf3b] add rax, rcx
|
||||
[0x2007ffbf3e] mov ecx, 0x06
|
||||
[0x2007ffbf43] add rax, rcx
|
||||
[0x2007ffbf46] mov ecx, 0x07
|
||||
[0x2007ffbf4b] int3 ;; oops! should probably patch this in the disassembly!
|
||||
- [0x2007ffbf4c] add eax, ecx
|
||||
[0x2007ffbf4e] mov eax, [r15+r14*1+0x08]
|
||||
[0x2007ffbf56] mov rcx, r14
|
||||
[0x2007ffbf59] add rcx, 0x00
|
||||
[0x2007ffbf60] cmp rax, rcx
|
||||
[0x2007ffbf63] jnz 0x0000002007FFBF19
|
||||
[0x2007ffbf69] mov eax, [r15+r14*1]
|
||||
[0x2007ffbf71] ret
|
||||
[0x2007ffbf72] add [rax], al
|
||||
[0x2007ffbf74] add [rax], al
|
||||
[0x2007ffbf76] add [rax], al
|
||||
[0x2007ffbf78] add [rax], al
|
||||
[0x2007ffbf7a] add [rax], al
|
||||
[0x2007ffbf7c] add [rax], al
|
||||
[0x2007ffbf7e] add [rax], al
|
||||
[0x2007ffbf80] in al, 0x08
|
||||
[0x2007ffbf82] INVALID (0x16)
|
||||
[0x2007ffbf82] add [rax], al
|
||||
[0x2007ffbf84] add [rcx], al
|
||||
[0x2007ffbf86] add [rbx], al
|
||||
[0x2007ffbf88] add [rax], al
|
||||
[0x2007ffbf8a] INVALID (0x00)
|
||||
|
||||
;; remove the breakpoint
|
||||
gcs> (:ubp #x2007ffbf4b)
|
||||
|
||||
;; continue, it stays running
|
||||
gcs> (:cont)
|
||||
gcr>
|
||||
|
||||
;; break and check, the code is back to normal!
|
||||
gcr> (:break)
|
||||
Target has stopped. Run (:di) to get more information.
|
||||
Read symbol table (146816 bytes, 124 reads, 123 symbols, 1.28 ms)
|
||||
rax: 0x0000000000000015 rcx: 0x0000000000000007 rdx: 0x0000000000000000 rbx: 0x0000002000000000
|
||||
rsp: 0x00007fddcde75c58 rbp: 0x00007fddcde75cc0 rsi: 0x0000000000000000 rdi: 0x0000000000000000
|
||||
r8: 0x0000000000147d24 r9: 0x0000002000000000 r10: 0x00007fddcde75ca0 r11: 0x0000000000000000
|
||||
r12: 0x0000000000147d24 r13: 0x0000002007ffbf14 r14: 0x0000000000147d24 r15: 0x0000002000000000
|
||||
rip: 0x0000002007ffbf4b
|
||||
[0x2007ffbf2b] add rax, rcx
|
||||
[0x2007ffbf2e] mov ecx, 0x04
|
||||
[0x2007ffbf33] add rax, rcx
|
||||
[0x2007ffbf36] mov ecx, 0x05
|
||||
[0x2007ffbf3b] add rax, rcx
|
||||
[0x2007ffbf3e] mov ecx, 0x06
|
||||
[0x2007ffbf43] add rax, rcx
|
||||
[0x2007ffbf46] mov ecx, 0x07
|
||||
- [0x2007ffbf4b] add rax, rcx
|
||||
[0x2007ffbf4e] mov eax, [r15+r14*1+0x08]
|
||||
[0x2007ffbf56] mov rcx, r14
|
||||
[0x2007ffbf59] add rcx, 0x00
|
||||
[0x2007ffbf60] cmp rax, rcx
|
||||
[0x2007ffbf63] jnz 0x0000002007FFBF19
|
||||
[0x2007ffbf69] mov eax, [r15+r14*1]
|
||||
[0x2007ffbf71] ret
|
||||
[0x2007ffbf72] add [rax], al
|
||||
[0x2007ffbf74] add [rax], al
|
||||
[0x2007ffbf76] add [rax], al
|
||||
[0x2007ffbf78] add [rax], al
|
||||
[0x2007ffbf7a] add [rax], al
|
||||
[0x2007ffbf7c] add [rax], al
|
||||
[0x2007ffbf7e] add [rax], al
|
||||
[0x2007ffbf80] in al, 0x08
|
||||
[0x2007ffbf82] INVALID (0x16)
|
||||
[0x2007ffbf82] add [rax], al
|
||||
[0x2007ffbf84] add [rcx], al
|
||||
[0x2007ffbf86] add [rbx], al
|
||||
[0x2007ffbf88] add [rax], al
|
||||
|
||||
gcs>
|
||||
|
||||
;; we can still properly exit from the target, even in this state!
|
||||
gcs> (e)
|
||||
Tried to reset a halted target, detaching...
|
||||
Error - target has timed out. If it is stuck in a loop, it must be manually killed.
|
||||
[Listener] Closed connection to target
|
||||
```
|
||||
@ -1,127 +0,0 @@
|
||||
# States in the Decompiler
|
||||
|
||||
## How can I tell if a file has states?
|
||||
Search in the `ir2.asm` file for `.type state`. If you see something like this:
|
||||
```
|
||||
.type state
|
||||
L28:
|
||||
.symbol plat-button-move-downward
|
||||
.symbol #f
|
||||
.symbol #f
|
||||
.symbol #f
|
||||
```
|
||||
that's a state, and you can expect the file to have `defstate`s.
|
||||
|
||||
## Virtual vs. Non-Virtual States
|
||||
A non-virtual state is stored in a global variable with the same name as the state. This is just like a global function. You don't have to do anything special with this - the decompiler will insert a `defstate` with the appropriate name and make sure that the name of the symbol and the name stored in the `state` itself match.
|
||||
|
||||
A virtual state is stored in the method table of a type, similar to a method. When doing a `go`, the virtual state will be looked up from the method table of the current process, allowing a child type of process to override the state of its parent. You can tell if a state is virtual by looking for a call to `inherit-state` or `method-set!`.
|
||||
|
||||
## Decompiling state handers
|
||||
Each state has up to 6 handler functions: `enter`, `post`, `exit`, `trans`, `code`, and `event`. In order for the decompiler to recognize these, you _must_ have the top-level function decompile. Do not attempt to decompile these functions until the top-level function passes.
|
||||
|
||||
The top-level analysis will find state handlers and name them appropriately. For non-virtual states, they will be named like `(<handler-name> <state-name>)`. Like `(code teetertotter-launch)`. For virtual states, the name will be `(<handler-name> <state-name> <type-name>)`. Use these names in type casts, stack structures, etc. These names will not work unless the top-level has been decompiled.
|
||||
|
||||
The type of the state handlers will be set up automatically by the type system, but requires that you get the type of the state itself correct.
|
||||
|
||||
Note: inside of `find_defstates.cpp` there is an option to enable rename prints that will print out the old name of the function before the rename.
|
||||
|
||||
## State Types
|
||||
Each state object must have its type set. The type of a state is:
|
||||
```
|
||||
(state <arg0> <arg1> ... <parent-type>)
|
||||
```
|
||||
|
||||
The args are the arguments given to the enter/code function. Both enter and code get the same arguments, and some may be unused, but this is ok.
|
||||
|
||||
The parent type is the type that the state belongs to. It must be `process` or a child of `process`. All state handlers are automatically behaviors of this type.
|
||||
|
||||
Here are two examples:
|
||||
```
|
||||
(define-extern silostep-rise (state symbol silostep))
|
||||
```
|
||||
will make all the `silostep-rise` handlers a behavior of `silostep` and will make `enter` and `code` take a single `symbol` argument.
|
||||
|
||||
## Go
|
||||
The `go` macro changes state. Internally it uses `enter-state`. Do not insert casts for `go`, it should work automatically.
|
||||
|
||||
TODO: there may be issues with the decompiler casting arguments to `go` - this is a bit tricky and I couldn't find a test case yet.
|
||||
|
||||
## Special Cases for Virtual States
|
||||
There are a few special cases for virtual states.
|
||||
|
||||
1. The state must be declared in the `deftype`, within the list of methods.
|
||||
2. The name of the method must be correct.
|
||||
3. Any `go` should be in a behavior of the appropriate process.
|
||||
|
||||
The next three sections explain these in more detail
|
||||
|
||||
### Declaring a virtual state
|
||||
Do not use `define-extern` to declare a virtual state. Instead, use the method list in the `deftype`.
|
||||
|
||||
As an example:
|
||||
```
|
||||
lui v1, L30 ;; [ 25] (set! gp-0 L30) [] -> [gp: state ]
|
||||
ori gp, v1, L30
|
||||
lw t9, inherit-state(s7)
|
||||
or a0, gp, r0
|
||||
lw v1, plat-button(s7)
|
||||
lwu a1, 104(v1)
|
||||
jalr ra, t9
|
||||
sll v0, ra, 0
|
||||
|
||||
lw t9, method-set!(s7)
|
||||
lw a0, sunken-elevator(s7)
|
||||
addiu a1, r0, 22
|
||||
or a2, gp, r0
|
||||
jalr ra, t9
|
||||
```
|
||||
|
||||
This means
|
||||
- The state object is `L30`.
|
||||
- The state is for type `sunken-elevator`.
|
||||
- The parent type of `sunken-elevator` should be `plat-button`
|
||||
- The method ID is 22.
|
||||
|
||||
The correct declaration should go under `:methods`. Like a normal method, it only needs to be defined in the parent type that first defines it. So we only have to put it in `plat-button` and can leave it out of `sunken-elevator`.
|
||||
|
||||
```
|
||||
(plat-button-pressed () _type_ :state 22)
|
||||
```
|
||||
|
||||
The first thing is the state name (described more in the next section). The next thing is a list of argument types given to the `enter` and `code` functions.
|
||||
|
||||
The ID should match the ID given to `method-set!` and it will be checked just like the normal method IDs.
|
||||
|
||||
If you get this wrong, you will get an error message like this:
|
||||
```
|
||||
virtual defstate attempted on something that isn't a state: (the-as state (method-of-type plat-button plat-button-move-upward))
|
||||
Did you forget to put :state in the method declaration?
|
||||
```
|
||||
which means that the `plat-button-move-upward` entry in `:methods` in `(deftype plat-button` is missing the `:state`
|
||||
|
||||
### Name of a virtual state
|
||||
The decompiler will check that the name in the method is correct. If you get it wrong, there will be an error that tells you the right name.
|
||||
|
||||
For example:
|
||||
```
|
||||
Disagreement between state name and type system name. The state is named plat-button-move-upward, but the slot is named dummy-24, defined in type plat-button
|
||||
```
|
||||
|
||||
This means you should rename `dummy-24` in `plat-button` to `plat-button-move-upward`.
|
||||
|
||||
|
||||
### Go to a virtual state
|
||||
You must be in a behavior in order for `go-virtual` to decompile successfully.
|
||||
|
||||
## The return value of `event` problem.
|
||||
There is one place that seems to rely on the return value of `event`. As a result, the default is to assume that `event` returns an `object`. However, there are sometimes `event` functions that clearly don't return a value and will refuse to decompile. The recommendation is:
|
||||
- Try to make all `event` functions return a value.
|
||||
- If it is absolutely not possible, make the function type return `none`, then the defstate should automatically insert a cast.
|
||||
|
||||
|
||||
## Unsupported
|
||||
Calls to `find-parent-method` that actually return a `state` will have type `function`. You must manually cast it. Make sure you get the argument types correct. This same problem exists for finding methods, but it has been very rare. If needed we can add special support in the decompiler/compiler to make this work.
|
||||
|
||||
If there is a function with multiple virtual `go`s which assume a different type at compile-time (accessing different parts of the type tree), then it is not possible to insert the right kind of cast yet.
|
||||
|
||||
@ -1,38 +0,0 @@
|
||||
## Drawable Trees
|
||||
At the highest level is the level file itself, which is a `bsp-header`. It contains a `drawable-tree-array`, which contains a small number of `drawable-tree`s. (~1 to 15?).
|
||||
|
||||
Each of these `drawable-tree`s is a different kind, distinguished by its type. Different types of trees go to different renderers. For example, there is a `drawable-tree-lowres-tfrag`, a `drawable-tree-instance-tie`, and even a `drawable-tree-actor`. It is possible to have multiple trees of the same type, as some trees seem to have a maximum size - for example a `drawable-tree-instance-tie` can have only 43 different prototypes, so it is common to see a few of these. These trees are the thing passed to the appropriate renderers. All the trees for all the levels get put in the `background-work` structure, which is then read by the renderers.
|
||||
|
||||
The `drawable-tree-tfrag` contains all the "normal" tfrag data. There are other trees like `trans`, `dirt`, `ice`, `lowres`, and `lowres-trans`, which are only present if the level actually has this type of geometry. As far as I can tell, the special ones follow the same format, but I haven't checked this in detail.
|
||||
|
||||
The `drawable-tree-tfrag` contains a small number (1 to 6?) of `drawable-inline-array`. They all refer to the same set of `tfragment`s (the actual things to draw), but have a different tree structure.
|
||||
|
||||
The `drawable-inline-array` contains `draw-node`s. Each `draw-node` contains between 0 and 8 children. The children are represented by a pointer to an inline array of children. The children can be either `draw-node`s or `tfragment`s. All the children are always the same type.
|
||||
|
||||
The first `drawable-inline-array` in the `drawable-tree-tfrag` is the full tree structure. It starts with some number of children that is smaller than 8. And from there on, all children are `draw-node` (always 8 or fewer children) or a `tfragment` directly. So this is the deepest possible tree. I believe the max depth seen is 6?
|
||||
|
||||
The next `drawable-inline-array` starts with a list of all nodes at with a depth of 1 from one of the top-level nodes of the first. So this has at most 64 entries. Like the previous tree, all the children from here on have 8 or fewer children.
|
||||
|
||||
This pattern continues. The n-th `drawable-inline-array` is a list of all nodes at depth n in the "real" tree.
|
||||
|
||||
There are two tricks to this:
|
||||
First, if the `drawable-inline-array` contains `draw-node`s, the type is actually `drawable-inline-array-node`. Unlike a `draw-node`, which can only contain 8 children, a `drawable-inline-array` can contain a huge number of children. The final `drawable-inline-array` is a list of a all children at the final depth, so it's always an array of `tfragment`. In this case, the type of the `drawable-inline-array` is a `drawable-inline-array-tfrag`, and it's just a giant list of all the `tfragment`s in the level.
|
||||
|
||||
The second trick is that the `draw-node`s and `tfragment`s are stored only once, even if they appear in multiple `drawable-inline-array`s. They used the weird "node = length + pointer to inline array of children" format and sorted nodes by depth to enable this.
|
||||
|
||||
|
||||
## Tfrag renderers
|
||||
The `tfrag-methods.gc` file has the methods of `drawable` that call the actual "draw" implementations in `tfrag.gc`. There is a near and "normal" version of the renderer. So far, it seems like trans/low-res, ice, dirt, etc don't have separate rendering code (or are part of the main `tfrag` program).
|
||||
|
||||
It looks possible for `tfragment`s to be drawn using the generic renderer, but so far I can't find the code where this happens.
|
||||
|
||||
## Tfrag data
|
||||
Tfrag also uses the `adgif-shader` idea. I believe the shaders are per-`tfragment` (though some may share, or there may be tricks to avoid resending the shader if consecutive `tfragment`s use the same settings).
|
||||
|
||||
I don't know if the `adgif-shader` is always 5 quadwords, like for sprite. It seems possible to use the `adgif-shader` just to set up texturing, but also have some other stuff. I believe that we currently log in these shaders and link to textures, so we can probably learn something from inspecting these.
|
||||
|
||||
There are 4 sets of data in `tfragment`: base, common, level0, level1. Each has its own DMA transfer (consisting of a address to data, plus a length). The details of which goes where is not clear yet. I _think_ that sometimes not all 4 are valid. There are only 3 start addresses stored, and the three DMA chains may be the same, or overlap.
|
||||
|
||||
The DMA data itself seems to only be loading data. It uses `unpack` (V4-16, V4-32, V3-32, V4-8) and `STROW`, `STMOD`, `STCYCL` to set up fancy unpacking tricks. No other VIFcodes have been found in any level.
|
||||
|
||||
Additionally, there are some color palettes that use the time-of-day system.
|
||||
@ -1,33 +0,0 @@
|
||||
# Editor Configuration
|
||||
|
||||
## EMacs
|
||||
|
||||
The following EMacs config file should get you started and configure OpenGOAL's formatting style
|
||||
|
||||
```lisp
|
||||
;; make gc files use lisp-mode
|
||||
(add-to-list 'auto-mode-alist '("\\.gc\\'" . lisp-mode))
|
||||
;; run setup-goal when we enter lisp mode
|
||||
(add-hook 'lisp-mode-hook 'setup-goal)
|
||||
|
||||
(defun setup-goal ()
|
||||
;; if we are in a gc file, change indent settings for GOAL
|
||||
(when (and (stringp buffer-file-name)
|
||||
(string-match "\\.gc\\'" buffer-file-name))
|
||||
(put 'with-pp 'common-lisp-indent-function 0)
|
||||
(put 'while 'common-lisp-indent-function 1)
|
||||
(put 'rlet 'common-lisp-indent-function 1)
|
||||
(put 'until 'common-lisp-indent-function 1)
|
||||
(put 'countdown 'common-lisp-indent-function 1)
|
||||
(put 'defun-debug 'common-lisp-indent-function 2)
|
||||
(put 'defenum 'common-lisp-indent-function 2)
|
||||
|
||||
;; indent for common lisp, this makes if's look nicer
|
||||
(custom-set-variables '(lisp-indent-function 'common-lisp-indent-function))
|
||||
(autoload 'common-lisp-indent-function "cl-indent" "Common Lisp indent.")
|
||||
;; use spaces, not tabs
|
||||
(setq-default indent-tabs-mode nil)
|
||||
)
|
||||
)
|
||||
|
||||
```
|
||||
@ -1,194 +0,0 @@
|
||||
# GOOS
|
||||
|
||||
GOOS is a macro language for GOAL. It is a separate language. Files written in GOAL end in `.gc` and files written in GOOS end in `.gs`. The REPL will display a `goos>` prompt for GOOS and `gc >` for GOAL.
|
||||
|
||||
There is a special namespace shared between GOOS and GOAL containing the names of the macros (written in GOOS) which can be used in GOAL code.
|
||||
|
||||
To access a GOOS REPL, run `(goos)` from the `gc >` prompt.
|
||||
|
||||
This document assumes some familiarity with the Scheme programming language. It's recommended to read a bit about Scheme first.
|
||||
|
||||
Note that most Scheme things will work in GOOS, with the following exceptions:
|
||||
- Scheme supports fractions, GOOS does not (it has separate integer/floating point types)
|
||||
- The short form for defining functions is `(desfun function-name (arguments) body...)`
|
||||
- GOOS does not have tail call optimization and prefers looping to recursion (there is a `while` form)
|
||||
|
||||
## Special Forms
|
||||
|
||||
Most forms in Scheme have a name, and list of arguments. Like:
|
||||
|
||||
```scheme
|
||||
(my-operation first-argument second-argument ...)
|
||||
```
|
||||
|
||||
Usually, each argument is evaluated, then passed to the operation, and the resulting value is returned. However, there are cases where not all arguments are evaluated. For example:
|
||||
|
||||
```scheme
|
||||
(if (want-x?)
|
||||
(do-x)
|
||||
(do-y)
|
||||
)
|
||||
```
|
||||
|
||||
In this case, only one of `(do-x)` and `(do-y)` are executed. This doesn't follow the pattern of "evaluate all arguments...", so it is a *SPECIAL FORM*. It's not possible for a function call to be a special form - GOOS will automatically evaluate all arguments. It is possible to build macros which act like special forms. There are some special forms built-in to the GOOS interpreter, which are documented in this section.
|
||||
|
||||
### define
|
||||
|
||||
This is used to define a value in the current lexical environment.
|
||||
For example:
|
||||
|
||||
```scheme
|
||||
(define x 10)
|
||||
```
|
||||
|
||||
will define `x` as a variable equal to `10` in the inner-most lexical environment. (Note, I'm not sure this is actually how Scheme works)
|
||||
|
||||
There is an optional keyword argument to pick the environment for definition, but this is used rarely. The only named environments are:
|
||||
- `*goal-env*`
|
||||
- `*global-env*`
|
||||
|
||||
Example:
|
||||
|
||||
```scheme
|
||||
(define :env *global-env* x 10)
|
||||
```
|
||||
|
||||
will define `x` in the global (outer-most) environment, regardless of where the `define` is written.
|
||||
|
||||
### quote
|
||||
This form simply returns its argument without evaluating it. There's a reader shortcut for this:
|
||||
|
||||
```scheme
|
||||
(quote x)
|
||||
|
||||
;; reader shortcut
|
||||
'x ;; same as (quote x)
|
||||
```
|
||||
|
||||
It's often used to get a symbol, but you can quote complex things like lists, pairs, and arrays.
|
||||
|
||||
```scheme
|
||||
goos> (cdr '(1 . 2))
|
||||
2
|
||||
goos> (cdr '(1 2))
|
||||
(2)
|
||||
goos> '#(1 2 3)
|
||||
#(1 2 3)
|
||||
```
|
||||
|
||||
### set!
|
||||
Set is used to search for a variable in the enclosing environments, then set it to a value.
|
||||
|
||||
```scheme
|
||||
(set! x (+ 1 2))
|
||||
```
|
||||
|
||||
will set the lexically closest `x` to 3. It's an error if there's no variable named `x` in an enclosing scope.
|
||||
|
||||
### lambda
|
||||
See any Lisp/Scheme tutorial for a good explanation of `lambda`.
|
||||
|
||||
Note that GOOS has some extensions for arguments. You can have a "rest" argument at the end, like this:
|
||||
|
||||
```scheme
|
||||
(lambda (a b &rest c) ...) ;; c is the rest arg
|
||||
(lambda (&rest a) ...) ;; a is the rest
|
||||
```
|
||||
|
||||
The rest argument will contain a list of all extra arguments passed to the function. If there are no extra arguments, the rest argument will be the empty list.
|
||||
|
||||
There are also keyword arguments, which contain a `&key` before the argument name.
|
||||
|
||||
```scheme
|
||||
(lambda (a b &key c) ...) ;; b is a keyword argument, a and c are not.
|
||||
(lambda (&key a &key b) ...) ;; a and b are keyword arguments
|
||||
```
|
||||
|
||||
These keyword arguments _must_ be specified by name. So to call the two functions above:
|
||||
|
||||
```scheme
|
||||
(f 1 2 :c 3) ;; a = 1, b = 2, c = 3
|
||||
(f :a 1 : b 2) ;; a = 1, b = 2
|
||||
```
|
||||
|
||||
Note that it is not required to put keyword arguments last, but it is a good idea to do it for clarity.
|
||||
|
||||
There are also keyword default arguments, which are like keyword arguments, but can be omitted from the call. In this case a default value is used instead.
|
||||
|
||||
```scheme
|
||||
(lambda (&key (c 12)) ...)
|
||||
(f :c 2) ;; c = 2
|
||||
(f) ;; c = 12
|
||||
```
|
||||
|
||||
The order of argument evaluation is:
|
||||
- All "normal" arguments, in the order they appear
|
||||
- All keyword/keyword default arguments, in alphabetical order
|
||||
- It is not recommended to rely on this
|
||||
- All rest arguments, in the order they appear
|
||||
|
||||
### cond
|
||||
Normal Scheme `cond`. If no cases matches and there is no `else`, returns `#f`.
|
||||
Currently `else` isn't implement, just use `#t` instead for now.
|
||||
|
||||
### or
|
||||
Short circuiting `or`. If nothing is truthy, `#f`. Otherwise returns first truthy.
|
||||
|
||||
### and
|
||||
Short circuiting `or`. If not all truthy, `#f`. Otherwise returns last truthy.
|
||||
|
||||
### macro
|
||||
Kind of like `lambda`, but for a macro.
|
||||
|
||||
A lambda:
|
||||
- Evaluate the arguments
|
||||
- Evaluate the body
|
||||
|
||||
A macro:
|
||||
- Don't evaluate the arguments
|
||||
- Evaluate the body
|
||||
- Evaluate that again
|
||||
|
||||
You can think about a `lambda` like a "normal" function, and a `macro` as a function that receive code as arguments (as opposed to values), and produces code as an output, which is then evaluated.
|
||||
|
||||
### quasiquote
|
||||
See any Lisp/Scheme tutorial. GOOS supports:
|
||||
|
||||
- `(quasiquote x)` or ``` `x ```
|
||||
- `(unquote x)` or `,x`
|
||||
- `(unquote-splicing x)` or `,@x`
|
||||
|
||||
### while
|
||||
Special while loop form for GOOS.
|
||||
|
||||
`(while condition body...)`
|
||||
|
||||
To add together `[0, 100)`:
|
||||
|
||||
```scheme
|
||||
(define count 0)
|
||||
(define sum 0)
|
||||
|
||||
(while (< count 100)
|
||||
(set! sum (+ sum count))
|
||||
(set! count (+ count 1))
|
||||
)
|
||||
|
||||
sum
|
||||
```
|
||||
|
||||
## Not Special Built-In Forms
|
||||
|
||||
TODO - None at this time
|
||||
|
||||
|
||||
## Namespace Details
|
||||
|
||||
The GOOS `define` form accepts an environment for definition. For example:
|
||||
|
||||
```scheme
|
||||
(define :env *goal-env* x 10)
|
||||
```
|
||||
|
||||
will define `x` in the `*goal-env*`. Any macros defined in the `*goal-env*` can be used as macros from within GOAL code.
|
||||
Things that aren't macros in the `*goal-env*` cannot be accessed.
|
||||
@ -1,128 +0,0 @@
|
||||
# Graphics
|
||||
|
||||
There are three frames in flight at a time:
|
||||
- One frame being processed by the CPU, building DMA lists. This is called the "calc frame"
|
||||
- One frame being processed by the GPU, going through the DMA lists and drawing triangles to one frame buffer. I will call this the "draw" frame.
|
||||
- One frame being displayed on the screen, stored in the other frame buffer. This is called the "on-screen" frame.
|
||||
|
||||
|
||||
|
||||
## Synchronization
|
||||
|
||||
The PC Port synchronizes on `syncv` and on `sync-path`. The `syncv` waits for an actual buffer swap and `sync-path` waits for the renderer to finish.
|
||||
|
||||
The game's code for this is kind of messy and confusing, and calls `sync-path` twice. On the PS2, you make sure rendering is done with `sync-path`, which waits for the DMA chain to finish. But they call this earlier than I think they need to, and I don't really understand why. I don't see any place where they read back from the finished frame or depth buffer. Or where they would overwrite the memory. There's a second call to `sync-path` right where you would expect, imeediately before the `syncv`. After `syncv`, they call some Sony library function to actually display the correct framebuffer, then immediately start sending the next DMA chain.
|
||||
|
||||
The stuff between `sync-path` and `syncv` is:
|
||||
- depth cue "calc" (seems fast)
|
||||
- screen filter "calc" (very fast, just DMA for a single quad)
|
||||
- calc for letterbox bars and blackout
|
||||
- debug draw
|
||||
- draw profiler (look at all the stuff that happens after!)
|
||||
- deci count draw
|
||||
- file info draw
|
||||
- stdcon text draw
|
||||
- iop/memcard info
|
||||
- tie buffer init
|
||||
- merc buffer init
|
||||
- post draw hook (does nothing)
|
||||
- bucket patching
|
||||
- cache flush
|
||||
- a second `sync-path`
|
||||
|
||||
According to the Performance Analyzer, this takes about 1% to 2% of a frame. They subtract off 4% of a frame from the profile bar so that 100% there is really around 96% of a frame, I guess to account for this extra time.
|
||||
|
||||
I'm really not sure why they have the first `sync-path` there. It makes some sense in debug mode so that you can draw the profile bar for the GPU after it has finished. Another theory is that they didn't want debug code and non-debug rendering running at the same time - the debug code will compete with the rendering to use the main bus, and will make the rendering slower. But it seems like you don't want this in the release version.
|
||||
|
||||
For now, the PC Port does sync on `sync-path`, but it probably doesn't need to.
|
||||
|
||||
## DMA Copy
|
||||
|
||||
Starting the main graphics DMA chain is intercepted by the PC Port code. In the GOAL thread, it iterates through the entire DMA chain and creates a copy. Then, it sends this copy to the graphics thread.
|
||||
|
||||
There are two reasons for copying:
|
||||
- If there is a memory bug in the game that corrupts the DMA buffer, it will not cause the renderer to crash. This is nice for debugging.
|
||||
- It will be easy to save this copied DMA chain to a file for debugging later.
|
||||
|
||||
The DMA copier attempts to reduce the amount of memory used. It divides the 128MB of RAM into 128 kB chunks, marks which ones contain DMA data, then only copies those chunks. The chunks are compacted and the DMA pointers are updated to point to the relocated chunks.
|
||||
|
||||
This operation might be expensive and we might need to get rid of it later. But it would be easy to get rid of - the renderers themselves just read from a DMA chain and don't care if it is a copy or not.
|
||||
|
||||
## DMA Chain Iteration
|
||||
|
||||
On the C++ side, you can iterate through DMA with the `DmaFollower` class. Here is an example of flattening a DMA chain to a sequence of bytes:
|
||||
```cpp
|
||||
std::vector<u8> flatten_dma() {
|
||||
DmaFollower state(memory, start_tag_addr);
|
||||
std::vector<u8> result;
|
||||
while (!state.ended()) {
|
||||
// tag value:
|
||||
state.current_tag();
|
||||
// tag address
|
||||
state.current_tag_offset();
|
||||
|
||||
// DMA data
|
||||
auto read_result = state.read_and_advance();
|
||||
// this is the transferred tag (u64 after dma tag, usually 2x vif tags)
|
||||
u64 tag = read_result.transferred_tag;
|
||||
|
||||
// the actual bytes (pointer to data in the input chain)
|
||||
result.insert(result.end(), read_result.data, read_result.data + read_result.size_bytes);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
```
|
||||
|
||||
This will take care of following `call` and `ret` and those details.
|
||||
|
||||
|
||||
## Buckets
|
||||
The game builds the DMA chain in 69 buckets. Each bucket corresponds to a rendering pass. In the OpenGLRenderer, you can designate a specific renderer for each bucket.
|
||||
|
||||
This can be set up in `init_bucket_renderers`, and you can pass arguments to the renderers:
|
||||
```cpp
|
||||
/*!
|
||||
* Construct bucket renderers. We can specify different renderers for different buckets
|
||||
*/
|
||||
void OpenGLRenderer::init_bucket_renderers() {
|
||||
|
||||
// create a DirectRenderer for each of these two buckets
|
||||
init_bucket_renderer<DirectRenderer>("debug-draw-0", BucketId::DEBUG_DRAW_0, 1024 * 8);
|
||||
init_bucket_renderer<DirectRenderer>("debug-draw-1", BucketId::DEBUG_DRAW_1, 1024 * 8);
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Each bucket renderer will have access to shared data. For now, this is textures and shaders.
|
||||
|
||||
## Textures
|
||||
Currently, the only textures supported are those uploaded with `upload-now!`, which does an immediate upload of the texture. The `TexturePool` receives the actual GOAL `texture-page` object, and uses the metadata there to figure out how to convert the texture to RGBA8888.
|
||||
|
||||
It maintains a lookup table of converted textures by TBP which is used for fast lookup in the renderers.
|
||||
|
||||
This system isn't great yet, I think we will need to improve it for time-of-day textures and textures that are swapped in and out of VRAM.
|
||||
|
||||
## Shaders
|
||||
The shaders themselves are stored in `opengl_renderer/shaders`. They should be named with `.frag` and `.vert` extensions.
|
||||
|
||||
To add a new shader, add it to `ShaderId` in `Shader.h` and give it a name in `Shader.cpp`. It will be compiled when the graphics system is loaded.
|
||||
|
||||
## Direct Renderer
|
||||
The direct renderer interprets GIF data directly. It is for rendering stuff that was set up entirely on the CPU, with no VU1 use.
|
||||
|
||||
Currently, the only known things that use this are:
|
||||
- debug stuff
|
||||
- font
|
||||
- screen filter/blackout/depth-cue
|
||||
- the progress menu
|
||||
|
||||
All of these cases are pretty simple, and this is nowhere near a full GS emulation.
|
||||
|
||||
It does a single pass through the DMA chain and creates arrays of triangles. It is designed to reduce the number of OpenGL draw calls when consecutive primitives are drawn in the same mode.
|
||||
|
||||
## Mysteries
|
||||
|
||||
Why did they put the first call to `sync-path` in?
|
||||
|
||||
How does the `upload-now!` during texture login for near textures work? It seems like it writes too much and might write over the other level's texture.
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 40 KiB |
1240
docs/markdown/lib.md
1240
docs/markdown/lib.md
File diff suppressed because it is too large
Load Diff
@ -1,147 +0,0 @@
|
||||
# OpenGOAL's Method System
|
||||
|
||||
OpenGOAL has a virtual method system. This means that child types can override parent methods. The first argument to a method is always the object the method is being called on, except for `new`.
|
||||
|
||||
All types have methods. Objects have access to all of their parents methods, and may override parent methods. All types have these 9 methods:
|
||||
|
||||
- `new` - like a constructor, returns a new object. It's not used in all cases, and on all types, and needs more documentation on when specifically it is used.
|
||||
- `delete` - basically unused, but like a destructor. Often calls `kfree`, which does nothing.
|
||||
- `print` - prints a short, one line representation of the object to the `PrintBuffer`
|
||||
- `inspect` - prints a multi-line description of the object to the `PrintBuffer`. Usually auto-generated by the compiler and prints out the name and value of each field.
|
||||
- `length` - Returns a length if the type has something like a length (number of characters in string, etc). Otherwise returns 0. Usually returns the number of filled slots, instead of the total number of allocated slots, when there is possibly a difference.
|
||||
- `asize-of` - Gets the size in memory of the entire object. Usually this just looks this up from the appropriate `type`, unless it's dynamically sized.
|
||||
- `copy` - Create a copy of this object on the given heap. Not used very much?
|
||||
- `relocate` - Some GOAL objects will be moved in memory by the kernel as part of the compacting actor heap system. After being moved, the `relocate` method will be called with the offset of the move, and the object should fix up any internal pointers which may point to the old location. It's also called on v2 objects loaded by the linker when they are first loaded into memory.
|
||||
- `memusage` - Not understood yet, but probably returns how much memory in bytes the object uses. Not supported by all objects.
|
||||
|
||||
Usually a method which overrides a parent method must have the same argument and return types. The only exception is `new` methods, which can have different argument/return types from the parent. (Dee the later section on `_type_` for another exception)
|
||||
|
||||
The compiler's implementation for calling a method is:
|
||||
- Is the type a basic?
|
||||
- If so, look up the type using runtime type information
|
||||
- Get the method from the vtable
|
||||
- Is the type not a basic?
|
||||
- Get the method from the vtable of the compile-time type
|
||||
- Note that this process isn't very efficient - instead of directly linking to the slot in the vtable (one deref) it first looks up the `type` by symbol, then the slot (two derefs). I have no idea why it's done this way.
|
||||
|
||||
In general, I suspect that the method system was modified after GOAL was first created. There is some evidence that types were once stored in the symbol table, but were removed because the symbol table became full. This could explain some of the weirdness around method calls/definition rules, and the disaster `method-set!` function.
|
||||
|
||||
All type definitions should also define all the methods, in the order they appear in the vtable. I suspect GOAL had this as well because the method ordering otherwise seems random, and in some cases impossible to get right unless (at least) the number of methods was specified in the type declaration.
|
||||
|
||||
## Special `_type_` Type
|
||||
|
||||
The first argument of a method always contains the object that the method is being called on. It also must have the type `_type_`, which will be substituted by the type system (at compile time) using the following rules:
|
||||
|
||||
- At method definition: replace with the type that the method is being defined for.
|
||||
- At method call: replace with the compile-time type of the object the method is being called on.
|
||||
|
||||
The type system is flexible with allowing you to use `_type_` in the method declaration in `deftype`, but not using `_type_` in the actual `defmethod`.
|
||||
|
||||
A method can have other arguments or a return value that's of type `_type_`. This special "type" will be replaced __at compile time__ with the type which is defining or calling the method. No part of this exists at runtime. It may seem weird, but there are two uses for this.
|
||||
|
||||
The first is to allow children to specialize methods and have their own child type as an argument type. For example, say you have a method `is-same-shape`, which compares two objects and sees if they are the same shape. Suppose you first defined this for type `square` with
|
||||
```
|
||||
(defmethod square is-same-shape ((obj1 square) (obj2 square))
|
||||
(= (-> obj1 side-length) (-> obj2 side-length))
|
||||
)
|
||||
```
|
||||
|
||||
Then, if you created a child class of `square` called `rectangle` (this is a terrible way to use inheritance, but it's just an example), and overrode the `is-same-shape` method, you would have to have arguments that are `square`s, which blocks you from accessing `rectangle`-specific fields. The solution is to define the original method with type `_type_` for the first two arguments. Then, the method defined for `rectangle` also will have arguments of type `_type_`, which will expand to `rectangle`.
|
||||
|
||||
|
||||
The second use is for a return value. For example, the `print` and `inspect` methods both return the object that is passed to them, which will always be the same type as the argument passed in. If `print` was define as `(function object object)`, then `(print my-square)` would lose the information that the return object is a `square`. If `print` is a `(function _type_ _type_)`, the type system will know that `(print my-square)` will return a `square`.
|
||||
|
||||
## Details on the Order of Overrides
|
||||
|
||||
The order in which you `defmethod` and `deftype` matters.
|
||||
|
||||
When you `deftype`, you copy all methods from the parent. When you `defmethod`, you always set a method in that type. You may also override methods in a child if: the child hasn't modified that method already, and if you are in a certain mode. This is a somewhat slow process that involves iterating over the entire symbol table and every type in the runtime, so I believe it was disabled when loading level code, and you just had to make sure to `deftype` and `defmethod` in order.
|
||||
|
||||
Assume you have the type hierarchy where `a` is the parent of `b`, which is the parent of `c`.
|
||||
|
||||
If you first define the three types using `deftype`, then override a method from `a` on `c`, then override that same method on `b`, then `c` won't use the override from `b`.
|
||||
|
||||
If you first define the three types using `deftype`, then override a method on `b`, it will _sometimes_ do the override on `c`. This depends on the value of the global variable `*enable-method-set*`, and some other confusing options. It may also print a warning but still do the override in certain cases.
|
||||
|
||||
## Built in Methods
|
||||
|
||||
All types have these 9 methods. They have reasonable defaults if you don't provide anything.
|
||||
|
||||
### `new`
|
||||
|
||||
The new method is a very special method used to construct a new object, like a constructor. Note that some usages of the `new` keyword __do not__ end up calling the new method. See the `new` section for more details. Unlike C++, fields of a type and elements in an array are not constructed either.
|
||||
|
||||
The first argument is an "allocation", indicating where the object should be constructed. It can be
|
||||
- The symbol `'global` or `'debug`, indicating the global or debug heaps
|
||||
- The symbols `'process-level-heap` or `'loading-level`, indicating whatever heaps are stored in those symbols.
|
||||
- `'process`, indicating the allocation should occur on the current process heap.
|
||||
- `'scratch`, for allocating on the scratchpad. This is unused.
|
||||
- Otherwise it's treated as a 16-byte aligned address and used for in place construction (it zeros the memory first)
|
||||
|
||||
The second argument is the "type to make". It might seem stupid at first, but it allows child classes to use the same `new` method as the parent class.
|
||||
|
||||
The remaining arguments can be used for whatever you want.
|
||||
|
||||
When writing your own `new` methods, you should ignore the `allocation` argument and use the `object-new` macro to actually do the allocation. This takes care of all the details for getting the memory (and setting up runtime type information if its a basic). See the section on `object-new` for more details.
|
||||
|
||||
### `delete`
|
||||
|
||||
This method isn't really used very much. Unlike a C++ destructor it's never called automatically. In some cases, it's repurposed as a "clean up" type function but it doesn't actually free any memory. It takes no arguments. The default implementations call `kfree` on what the allocation, but there are two issues:
|
||||
1. The implementation is sometimes wrong, likely confusing doing pointer math (steps by array stride) with address math (steps by one byte).
|
||||
2. The `kfree` function does nothing.
|
||||
|
||||
The `kheap` system doesn't really support freeing objects unless you free in the opposite order you allocate, so it makes sense that `delete` doesn't really work.
|
||||
|
||||
### `print`
|
||||
|
||||
This method should print out a short description of the object (with no newlines) and return the object. The printing should be done with `(format #t ...)` (see the section on `format`) for more information. If you call `print` by itself, it'll make this description show up in the REPL. (Note that there is some magic involved to add a newline here... there's actually a function named `print` that calls the `print` method and adds a newline)
|
||||
|
||||
The default short description looks like this: `#<test-type @ #x173e54>` for printing an object of type `test-type`. Of course, you can override it with a better version. Built-in types like string, type, boxed integer, pair, have reasonable overrides.
|
||||
|
||||
This method is also used to print out the object with `format`'s `~A` format option.
|
||||
|
||||
### `inspect`
|
||||
|
||||
This method should print out a detailed, multi-line description. By default, `structure`s and `basic`s will have an auto-generated method that prints out the name and value of all fields. For example:
|
||||
|
||||
```lisp
|
||||
gc > (inspect *kernel-context*)
|
||||
[00164b44] kernel-context
|
||||
prevent-from-run: 65
|
||||
require-for-run: 0
|
||||
allow-to-run: 0
|
||||
next-pid: 2
|
||||
fast-stack-top: 1879064576
|
||||
current-process: #f
|
||||
relocating-process: #f
|
||||
relocating-min: 0
|
||||
relocating-max: 0
|
||||
relocating-offset: 0
|
||||
low-memory-message: #t
|
||||
```
|
||||
|
||||
In some cases this method is overridden to provide nicer formatting.
|
||||
|
||||
### `length`
|
||||
|
||||
This method should return a "length". The default method for this just returns 0, but for things like strings or buffers, it could be used to return the number of characters or elements in use. It's usually used to refer to how many are used, rather than the capacity.
|
||||
|
||||
### `asize-of`
|
||||
|
||||
This method should return the size of the object. Including the 4 bytes of type info for a `basic`.
|
||||
|
||||
By default this grabs the value from the object's `type`, which is only correct for non-dynamic types. For types like `string` or other dynamic types, this method should be overridden. If you intend to store dynamically sized objects of a given type on a process heap, you __must__ implement this method accurately.
|
||||
|
||||
### `copy`
|
||||
|
||||
Creates a copy of the object. I don't think this used very much. Just does a `memcpy` to duplicate by default.
|
||||
|
||||
### `relocate`
|
||||
|
||||
The exact details are still unknown, but is used to update internal data structures after an object is moved in memory. This must be support for objects allocated in process heaps of processes allocated on the actor heap or debug actor heap.
|
||||
|
||||
It's also called on objects loaded from a GOAL data object file.
|
||||
|
||||
### `mem-usage`
|
||||
|
||||
Not much is known yet, but used for computing memory usage statistics.
|
||||
@ -1,85 +0,0 @@
|
||||
# Object File Formats
|
||||
|
||||
## CGO/DGO Files
|
||||
|
||||
The CGO/DGO file format is exactly the same - the only difference is the name of the file. The DGO name indicates that the file contains all the data for a level. The engine will load these files into a level heap, which can then be cleared and replaced with a different level.
|
||||
|
||||
I suspect that the DGO file name came first, as a package containing all the data in the level which can be loaded very quickly. Names in the code all say `dgo`, and the `MakeFileName` system shows that both CGO and DGO files are stored in the `game/dgo` folder. Probably the engine and kernel were packed into a CGO file after the file format was created for loading levels.
|
||||
|
||||
Each CGO/DGO file contains a bunch of individual object files. Each file has a name. There are some duplicate names - sometimes the two files with the same names are very different (example, code for an enemy, art for an enemy), and other times they are very similar (tiny differences in code/data). The files come in two versions, v4 and v3, and both CGOs and DGOs contain both versions. If an object file has code in it, it is always a v3. It is possible to have a v3 file with just data, but usually the data is pretty small. The v4 files tend to have a lot of data in them. My theory is that the compiler creates v3 files out of GOAL source code files, and that other tools for creating things like textures/levels/art-groups generate v4 objects. There are a number of optimizations in the loading process for v4 objects that are better suited for larger files. To stay at 60 fps always, a v3 object must be smaller than around 750 kB. A v4 object does not have this limitation.
|
||||
|
||||
### The V3 format
|
||||
|
||||
The v3 format is divided into three segments:
|
||||
1. Main: this contains all of the functions/data that will be used by the game.
|
||||
2. Debug: this is only loaded in debug mode, and is always stored on a separate `kdebugheap`.
|
||||
3. Top Level: this contains some initialization code to add functions/variables to the symbol table, and any user-written initialization. It runs once, immediately after the object is loaded, then is thrown away.
|
||||
|
||||
Each segment also has linking data, which tells the linker how to link references to symbols, types, and memory (possibly in a different segment).
|
||||
|
||||
This format will be different between the PC and PS2 versions, as linking data for x86-64 will need to look different from MIPS.
|
||||
|
||||
Each segment can contain functions and data. The top-level segment must start with a function which will be run to initialize the object. All the data here goes through the GOAL compiler and type system.
|
||||
|
||||
### The V4 format
|
||||
|
||||
The V4 format contains just data. Like v3, the data is GOAL objects, but was probably generated by a tool that wasn't the compiler. A V4 object has no segments, but must start with a `basic` object. After being linked, the `relocate` method of this `basic` will be called, which should do any additional linking required for the specific object.
|
||||
|
||||
Because this is just data, there's no reason for the PC version to change this format. This means we can also check the
|
||||
|
||||
Note: you may see references to v2 format in the code. I believe v4 format is identical to v2, except the linking data is stored at the end, to enable a "don't copy the last object" optimization. The game's code uses the `work_v2` function on v4 objects as a result, and some of my comments may refer to v2, when I really mean v4. I believe there are no v2 objects in any games.
|
||||
|
||||
### The Plan
|
||||
|
||||
- Create a library for generating obj files in V3/V4 format
|
||||
- V4 should match game exactly. Doesn't support code.
|
||||
- V3 is our own thing. Must support code.
|
||||
|
||||
We'll eventually create tools which use the library in V4 mode to generate object files for rebuilding levels and textures. We may need to wait until more about these formats is understood before trying this.
|
||||
|
||||
The compiler will use the library in V3 mode to generate object files for each `gc` (GOAL source code) file.
|
||||
|
||||
### CGO files
|
||||
|
||||
The only CGO files read are `KERNEL.CGO` and `GAME.CGO`.
|
||||
|
||||
The `KERNEL.CGO` file contains the GOAL kernel and some very basic libraries (`gcommon`, `gstring`, `gstate`, ...). I believe that `KERNEL` was always loaded on boot during development, as its required for the Listener to function.
|
||||
|
||||
The `GAME.CGO` file combines the contents of the `ENGINE`, `COMMON` and `ART` CGO files. `ENGINE` contains the game engine code, `COMMON` contains level-specific code (outside of the game engine) that is always loaded. If code is used in basically all the levels, it makes sense to put in in `COMMON`, so it doesn't have to be loaded for each currently active level. The `ART` CGO contains common art/textures/models, like Jak and his animations.
|
||||
|
||||
The `JUNGLE.CGO`, `MAINCAVE.CGO`, `SUNKEN.CGO` file contains some copies of files used in the jungle, cave, LPC levels. Some are a tiny bit different. I believe it is unused.
|
||||
|
||||
The `L1.CGO` file contains basically all the level-specific code/Jak animations and some textures. It doesn't seem to contain any 3D models. It's unused, but I'm still interested in understanding its format, as the Jak 1 demos have this file.
|
||||
|
||||
The `RACERP.CGO` file contains (I think) everything needed for the Zoomer. Unused. The same data appears in the levels as needed, maybe with some slight differences.
|
||||
|
||||
The `VILLAGEP.CGO` file contains common code shared in village levels, which isn't much (oracle, warp gate). Unused. The same data appears in the levels as needed.
|
||||
|
||||
The `WATER-AN.CGO` file contains some small code/data for water animations. Unused. The same data appears in the levels as needed.
|
||||
|
||||
### CGO/DGO Loading Process
|
||||
|
||||
A CGO/DGO file is loaded onto a "heap", which is just a chunk of contiguous memory. The loading process is designed to be fast, and also able to fill the entire heap, and allow each object to allocate memory after it is loaded. The process works like this:
|
||||
|
||||
1. Two temporary buffers are allocated at the end of the heap. These are sized so that they can fit the largest object file, not including the last object file.
|
||||
2. The IOP begins loading, and is permitted to load the first two object files to the two temporary buffers
|
||||
3. The main CPU waits for the first object file to be loaded.
|
||||
4. While the second object file being loaded, the first object is "linked". The first step to linking is to copy the object file data from the temporary buffer to the bottom of the heap, kicking out all the other data in the process. The linking data is checked to see if it is in the top of the heap, and is moved there if it isn't already. The run-once initialization code is copied to another temporary allocation on top of the heap and the debug data is copied to the debug heap.
|
||||
5. Still, while the second object file is being loaded, the linker runs on the first object file.
|
||||
6. Still, while the second object file is being loaded, the second object's initialization code is run (located in top of the heap). The second object may allocate from this heap, and will get valid memory without creating gaps in the heap.
|
||||
7. Memory allocated from the top during linking is freed.
|
||||
8. The IOP is allowed to load into the first buffer again.
|
||||
9. The main CPU waits for the second object to be loaded, if the IOP hasn't finished yet.
|
||||
10. This double-buffering pattern continues - while one object is loaded into a buffer, the other one will be copied to the bottom of the heap, linked, and initialized. When the second to last object is loaded, the IOP will wait an extra time until the main CPU has finished linking it until loading the last object (one additional wait) because the last object has a special case.
|
||||
11. The last object will be loaded directly onto the bottom of the heap, as there may not be enough memory to use the temporary buffers and load the last object. The temporary buffers are freed.
|
||||
12. If the last object is a v3, its linking data will be moved to the top-level, and the object data will be moved to fill in the gap left behind. If the last object is a v2, the main data will be at the beginning of the object data, so there is an optimization that will avoid copying the object data to save time, if the data is already close to being in the right place.
|
||||
|
||||
Generally the last file in a level DGO will be the largest v4 object. You can only have one file larger than a temporary buffer, and it must come last. The last file also doesn't have to be copied after being loaded into memory if it is a v4.
|
||||
|
||||
#### V3 max size
|
||||
|
||||
A V3 object is copied all at once with a single `ultimate-memcpy`. Usually linking gets to run for around 3 to 5% of a total frame. The `ultimate-memcpy` routine does a to/from scratchpad transfer. In practice, mem/spr transfers are around 1800 MB/sec, and the data has to be copied twice, so the effective bandwidth is 900 MB/sec.
|
||||
|
||||
`900 MB / second * (0.04 * 0.0167 seconds) = 601 kilobytes`
|
||||
|
||||
This estimate is backed up by the the chunk size of the v4 copy routine, which copies one chunk per frame. It picks 524 kB as the maximum amount that's safe to copy per frame.
|
||||
@ -1,355 +0,0 @@
|
||||
# Porting Tfrag
|
||||
Tfrag is the renderer for non-instanced background geometry. It's typically used for the floor and unique walls/level geometry. It has a level of detail system, and time of day lighting, optionaly transparancy and that's it. No other features.
|
||||
|
||||
The approach I took was to go slowly and understand the rendering code. I made two different "test" renderers that were slow but did things exactly the same way as in the PS2 version. After this, I made a custom PC version called tfrag3. The key difference with tfrag3 is that there is an offline preprocessing step that reads the level data and outputs data in a good format for PC.
|
||||
|
||||
Trying to understand the rendering code is annoying, but I think it was worth it in the end.
|
||||
- you can often leave out huge chunks of code. I never touched `tfrag-near` or most of the `tfrag` VU program.
|
||||
- you can eventually figure out how move more to the GPU. For example, all clipping/scissoring and transformation is done on the GPU. This is faster (GPUs are good) and easier (OpenGL does it automatically if you set it up right, you don't have to do the math).
|
||||
- you can rearrange things for better performance. Keeping the number of OpenGL draw calls down is probably the best thing we can do for performance.
|
||||
|
||||
But in order to understand the renderer, I had to start with a slow "emulation-like" port.
|
||||
|
||||
This document is divided into three parts:
|
||||
1. PS2 rendering (in Jak).
|
||||
2. Jak's `drawable` system
|
||||
3. Tfrag-specific details
|
||||
|
||||
# Basics of PS2 Rendering
|
||||
The main idea of the Jak rendering system is that there are always two frames in progress at a time. One frame is being "rendered" meaning triangles are being transformed and rasterized and the VRAM is being written to. The other frame is being "calculated", meaning the game is building the list of instructions to draw the frame. The "calculation" happens from GOAL code, mostly on the EE, and builds a single giant "DMA chain". At the end of a frame, the engine takes the full DMA chain that was built, and sends it to the rendering hardware. The rendering process is all "automatic" - once it gets the list of data it will run for an entire frame and do all of the drawing.
|
||||
|
||||
The EE user manual sections for DMAC, VPU (VU), and GIF are worth reading before trying to understand new rendering code.
|
||||
|
||||
Because the calculation and rendering happen simultaneously, the calculation cannot use the same hardware and memory as the rendering. The following resources are used only by rendering:
|
||||
- VU1
|
||||
- VIF1
|
||||
- GIF
|
||||
- GS
|
||||
|
||||
The following resources are used only by calculation:
|
||||
- VU0-macro mode (`vf` register on EE and `vadd` like instructions)
|
||||
- VU0-micro mode
|
||||
- VIF0
|
||||
- The scratchpad
|
||||
|
||||
The following resources are shared:
|
||||
- DMA controller. It can handle multiple transfers to different places at the same time, and there are no shared destinations, so there's no issue here. Rendering uses only to VIF1. Calculation uses to VIF0, to scratchpad, and from scratchpad.
|
||||
- The "global" DMA buffer. The calculation process fills this buffer and the rendering reads from it. There are two copies of this buffer and the engine will swap them automatically at the end of the frame. So one copy is always being filled while the other is being read and the graphics code mostly doesn't worry about this.
|
||||
- DMA data inside of level data. The DMA list may include chunks of data that's part of the level data. In practice there's not much to be aware of here - the rendering process just reads this data.
|
||||
|
||||
## DMA
|
||||
The whole rendering system is driven by DMA. The DMA controller can copy data from main memory to different peripherals. If the destination is "busy", it will wait. So it doesn't just blindly dump data into things as fast as it can - it only sends data if the destination is ready to accept it. The DMA controller is controlled by "DMA tags". These contain a command like "transfer X bytes of data from address Y, then move on to the DMA tag at address Z". This allows the game to build up really complicated linked-lists of data to send.
|
||||
|
||||
The DMA list built for rendering is divided into buckets. See `dma-h.gc` for the bucket names. Individual renderers add data to buckets, and then the buckets are linked together in the order they are listed in that enum. Code like `tfrag` won't start DMA to VIF1 or deal with linking buckets - that is handled by the game engine.
|
||||
|
||||
However, code like `tfrag` may just set up its own transfers to/from SPR and VIF0 - these are free-to-use during the "calculation" step.
|
||||
|
||||
## The VIF
|
||||
The VIF is "vector unit interface". There's one for VU0 and VU1 and they are (as far as I know) identical. The rendering DMA list is sent directly to VIF1. There are also "tags" that control the VIF. The general types of tags are:
|
||||
- "take the following N bytes of data and copy it to VU data memory" (possibly with some fancy "unpacking" operation to move stuff around)
|
||||
- "take the following N bytes of data and copy it to VU program memory" - to upload a new VU program
|
||||
- "run a VU program starting at this address in VU program memory"
|
||||
- "send this data **direct**ly to the GIF" This is called a "direct" transfer. It's typically used to set things up on the GS that will be constant for one specific renderer.
|
||||
|
||||
I haven't seen VIF0 really used much. The pattern for VIF1 is usually:
|
||||
1. upload program
|
||||
2. upload some constants
|
||||
3. upload some model data
|
||||
4. run program
|
||||
5. repeat steps 3 and 4 many times
|
||||
|
||||
## VU programs
|
||||
The VU programs are usually responsible for transforming vertices with the camera matrix, clipping, lighting calculations, etc. The output of a VU program is GIF packets containing the actual drawing information (transformed vertices, drawing settings, etc).
|
||||
|
||||
The usual pattern is that the VU program will build up a GIF packet, then use the `XGKICK` instruction with the address of the start of the packet. This will start transferring the packet directly to the GIF. The transfer happens in the background. The transfer will only be completed once all triangles are drawn - there's no buffer on the GIF/GS. A single packet can be pretty big and have many triangles.
|
||||
|
||||
For the tfrag1/tfrag2 renderers, I ported up to this part. Then, I sent the `xgkick` data to the `DirectRenderer` which can handle this format of data. It is not super fast, but it's nice for debugging. Being able to inspect this was helpful to understand how it works.
|
||||
|
||||
## VU buffer hell
|
||||
Typically the VU programs have 4 buffers. There is a buffer for input and output data, and both are double buffered. This allows you to be uploading new data with DMA, transforming vertices, and sending data to the GIF all at the same time.
|
||||
|
||||
All 4 buffers are in use at the same time.
|
||||
1. Untransformed Data being uploaded from the DMA list to VU data. This happens automatically by DMA.
|
||||
2. Untransformed Data being transformed by the VU1 program.
|
||||
3. A GIF packet being built from the output of the transformation. This is written by the VU1 program.
|
||||
4. A GIF packet currently being `XGKICK`ed.
|
||||
|
||||
Once 1 is full and 2 is totally used, these buffers are swapped. The same thing happens for 3 and 4.
|
||||
In some renderers, these swaps are always done at the same time. For example `sprite`. This tends to use the built-in `xitop` instructions for managing double buffering.
|
||||
|
||||
In other renderers, the buffer swaps can happen at different times. This leads to awful code where you have 4 different versions of the same renderer for all possible combinations of which buffers are input/output. Storing the address of the input/output buffer in a variable can lead to extra instructions inside the transformation loops, which will significantly slow down.
|
||||
|
||||
## GIF
|
||||
The GIF can receive commands like:
|
||||
- "set the alpha blending mode to X"
|
||||
- "use texture located at VRAM address Y"
|
||||
- "draw a triangle"
|
||||
|
||||
It can't do any transformation or lighting calculations.
|
||||
|
||||
|
||||
# Jak `drawable` system
|
||||
There is a `drawable` system that's used to store things that can be "drawn". It uses a tree structure. So you can do something like
|
||||
```
|
||||
(draw some-level)
|
||||
```
|
||||
and it will recursively go through the entire level's tree of drawables and draw everything. Note that there are a lot of tricks/hacks so not every `drawable` supports `draw` and some code may defer some `draw`s until later.
|
||||
|
||||
The lowest-level drawable for tfrag is `tfragment`. It makes sense to split up the level into "fragments" because the entire level is way too big to fit in the VU memory. Most of the time, you can't see every triangle in the level, so it makes sense to skip uploading the fragments that you know can't be seen.
|
||||
There are thousands of these fragments. They tend to be ~ a few kB and each contains a chunk of data to upload to the VUs. Note that `draw` is not called directly on `tfragment`, despite the fact that they are `drawable`s (more details later). The `tfragment` is just a reference to some DMA data.
|
||||
|
||||
## Drawables for Tfrag
|
||||
|
||||
The top-level drawable type for an entire level is `bsp-header`. This is the type of the `-vis` file of the level's DGO, and has all the graphics data (not including textures). It is also a `drawable`.
|
||||
|
||||
Within `bsp-header` is a `drawable-tree-array`. As the name implies, this contains an array of `drawable-tree`. Usually there are 5-10 of these in a level. There will be a `drawable-tree` for each renderer. Or possibly a few per renderer, if that renderer supports different modes. For example there's one for tfrag, one for transparent tfrag, one for tie, etc.
|
||||
|
||||
You can just check the type of the `drawable-tree` to see if it's for tfrag/tie etc. The tfrag types are:
|
||||
- `drawable-tree-tfrag` (parent of the rest)
|
||||
- `drawable-tree-trans-tfrag`
|
||||
- `drawable-tree-dirt-tfrag`
|
||||
- `drawable-tree-ice-tfrag`
|
||||
- `drawable-tree-lowres-tfrag`
|
||||
- `drawable-tree-lowres-trans-tfrag`
|
||||
|
||||
Each "tree" contains a bunch of `tfragment`s and a time of day color palette. But they are stored in a really weird way. There is a bounding volume hierarchy of `tfragment`s. This is just a tree-like structure where each node stores a sphere, and all the node's children fit inside of that sphere. The nodes at each depth are stored in an array. The layout of this tree is designed to let them use some crazy assembly SIMD 8-at-a-time traversal of the tree, with minimal pointer-chasing and good memory access patterns.
|
||||
|
||||
Each tree has an array of `drawable-inline-array`s, storing all the nodes at a given depth. The last `drawable-inline-array` is actually a `drawable-inline-array-frag`, which is a wrapper around an inline array of `tfragment`s.
|
||||
|
||||
The other arrays are used to store tree nodes to organize these `tfragment`s. Each node in the tree contains a `bsphere`. All tfrags below the node fit into this sphere.
|
||||
|
||||
The second to last array (if it exists) is a `drawable-inline-array-node`. This contains an inline array of `draw-node`. Each `draw-node` is the parent of between 1 and 8 `tfragment`s. They store a reference to the first child `tfragment` and a child count, and the children are just the ones that come after the first `tfragment` in memory.
|
||||
|
||||
The third to last array (if it exists) is also a `drawable-inline-array-node`, containing an inline array of `draw-node`. Each `draw-node` is the parent of between 1 and 8 `draw-node`s from the array mentioned above. They store a reference to the first child `draw-node` and a child count, and the children are stored consecutively.
|
||||
|
||||
This pattern continues until you get a `drawable-inline-array-node` with 8 or fewer nodes at the top.
|
||||
|
||||
All the `draw-node` and `tfragment`s have ID numbers. These are used for the occlusion culling system. The visibility numbering is shared with all the other `drawable-tree`s in the `bsp-header`. The indices are given out consecutively, starting from the roots. Between depths, they are aligned to 32 elements, so there are some unused ids. These IDs are the index of the bit in the visibility string.
|
||||
|
||||
With that out of the way, we can now go through the tfrag renderer
|
||||
|
||||
# Tfrag
|
||||
|
||||
The rough process for rendering is:
|
||||
- "login" the data
|
||||
- do "drawing" as part of the `drawable` system (on EE)
|
||||
- do the real "draw"
|
||||
- do culling
|
||||
- compute time of day colors (or other precomputation)
|
||||
- generate DMA lists
|
||||
- unpack data to VU memory
|
||||
- transform vertices
|
||||
- clip
|
||||
- build gs packets
|
||||
- XGKICK
|
||||
|
||||
I expect that most other renderers will be pretty similar.
|
||||
|
||||
## Login
|
||||
The tfrag data needs to be initialized before it can be used. You only have to do this once. This is called `login`, and it's a method of all `drawable`s. The level loader will call the `login` method of many things as part of the level load. For `tfrag`, all I had to do was decompile the `login` methods, and it worked and I could completely ignore this until tfrag3.
|
||||
|
||||
It's possible to just call `login` on an entire level, but this probably takes too long, so the level loader will cleverly split it up over multiple frames.
|
||||
|
||||
It is from:
|
||||
- `level-update`
|
||||
- `load-continue`
|
||||
- `level-update-after-load`
|
||||
- various calls to `login`.
|
||||
|
||||
|
||||
In the end, the only thing the `login` does for tfrag is:
|
||||
```
|
||||
(adgif-shader-login-no-remap (-> obj shader i))
|
||||
```
|
||||
for all the "shaders" in all the tfrags. A "shader" is an `adgif-shader`, which is just some settings for the GS that tells it drawing modes, like which texture to use, blending modes, etc. The `tfrag` VU1 code will send these to the GIF as needed when drawing. A `tfragment` can have multiple shaders. There is a different shader per texture.
|
||||
|
||||
The actual "shader" object is just 5x quadwords that contain "adress + data" format data. The address tells the GIF which parameter to change and the "data" has the value of the parameter. Some of them are not set properly in the level data, and the `adgif-shader-login-no-remap` function updates them. For tfrag, the 5 addresses are always the same:
|
||||
|
||||
- `TEST_1`: this sets the alpha and z test settings. This is set properly in the level data and `login` doesn't touch it.
|
||||
- `TEX0_1`: this has some texture parameters. This is 0 in the level data and is modified by `login`.
|
||||
- `TEX1_1`: this has more texture parameters. In the level data this is has `0x120` as the value and the address is set to the texture ID of the texture. During `login`, the texture ID is looked up in the texture pool and `TEX0_1`/`TEX1_1` are set to point to the right VRAM address and have the right settings to use the texture.
|
||||
- `MIPTBP1_1`: is mipmap settings. I ignore these because we do our own mipmapping.
|
||||
- `CLAMP_1`: this has texture clamp settings. This is set properly in the level data.
|
||||
- `ALPHA_1`: this has alpha blend settings. This is set properly in the level data.
|
||||
|
||||
|
||||
## Calling the `draw` method
|
||||
The `tfragment` at least pretends to use the `drawable` system, and the drawing is initiated by calling `draw` on the `drawable-tree-tfrag`. Getting this to actually be called took some digging - it uses some functions in later files that we haven't completed yet.
|
||||
|
||||
When the level is loaded, the `bsp-header` is added to the `*background-draw-engine*` by the level loader. The path to calling draw is:
|
||||
- In `main.gc`, there is a `display-loop`. This has a while loop that runs once per frame and runs many systems.
|
||||
- The `display-loop` calls `*draw-hook*`
|
||||
- The `*draw-hook*` variable is set to `main-draw-hook`
|
||||
- The `main-draw-hook` calls `real-main-draw-hook`
|
||||
- The `real-main-draw-hook` calls `(execute-connections *background-draw-engine*`
|
||||
- This "engine" calls the `add-bsp-drawable` function on the `bsp-header` for each loaded level.
|
||||
- The `draw` method of `bsp-header` sets up some stuff on the scratchpad and some `vf` registers.
|
||||
- The `draw` method of `bsp-header` calls `draw` on the `drawable-tree-array` (defined in parent class `drawable-group`)
|
||||
- The `draw` method of `drawable-group` checks if the level is visible, and if so calls `draw` on each tree.
|
||||
- The `draw` method of `drawable-tree-tfrag` simply adds the tree to a list of trees in `*background-work*`.
|
||||
|
||||
|
||||
## Real "drawing"
|
||||
Later on, in the `real-main-draw-hook`, there is a call to `finish-background`.
|
||||
|
||||
There's some stuff at the top of this function that's only used for the separate shrubbery renderer. It sets up some VU0 programs. I noticed that the stuff before tfrag drawing would overwrite this VU0 stuff so I ignored it for now.
|
||||
|
||||
The first thing that happens before any tfrag drawing is setting the `vf` registers to store the `math-camera` values. In OpenGOAL, the `vf` registers aren't saved between functions, so I had to manually use the `with-vf` macro with the `:rw 'write` flag to save these:
|
||||
```lisp
|
||||
(let ((v1-48 *math-camera*))
|
||||
(with-vf (vf16 vf17 vf18 vf19 vf20 vf21 vf22 vf23 vf24 vf25 vf26 vf27 vf28 vf29 vf30 vf31)
|
||||
:rw 'write
|
||||
(.lvf vf16 (&-> v1-48 plane 0 quad))
|
||||
(.lvf vf17 (&-> v1-48 plane 1 quad))
|
||||
;; ...
|
||||
```
|
||||
these will later be used in part of the drawing function. The `:rw 'write` flag will save these to a structure where we can read them later.
|
||||
|
||||
Then, for each tree:
|
||||
```
|
||||
(upload-vis-bits s1-0 gp-1 a2-4)
|
||||
```
|
||||
this uploads the visibility data to the scratchpad. The visibility data is stored at the end of the 16 kB scratchpad. The drawable with ID of `n` can look at the `n`-th bit of this data to determine if it is visible. The visibility IDs are per level, and the drawing order of the `tfrag` will alternate between levels, so they upload this for each tree to draw. It seems like you could skip this after the first upload if you detect that you're drawing multiple trees in the same level. They do it for TIE and not TFRAG and don't know why. The visibility data is based on the position of the camera. Currently this doesn't work so I modified it to upload all 1's.
|
||||
|
||||
The modification to the code to use the scratchpad in OpenGOAL is:
|
||||
```
|
||||
;;(spad-vis (the-as (pointer uint128) (+ #x38b0 #x70000000)))
|
||||
(spad-vis (scratchpad-ptr uint128 :offset VISIBLE_LIST_SCRATCHPAD))
|
||||
```
|
||||
The `0x38b0` offset is just something we've noticed over time as being the location of the visible list, so there's a constant I made for it. (TODO: I think it's also `terrain-context work background vis-list`)
|
||||
|
||||
The hack for visibility is:
|
||||
```lisp
|
||||
;; TODO this is a hack.
|
||||
(quad-copy! (-> arg0 vis-bits) (-> arg2 all-visible-list) (/ (+ (-> arg2 visible-list-length) 15) 16))
|
||||
```
|
||||
which actually modifies the level to say that everything is visbile. The `all-visible-list` is just a list which has `1` for every drawable that actually exists (I think, need to configm). There are some skipped ID's.
|
||||
|
||||
|
||||
The next part of drawing is:
|
||||
```
|
||||
(when (not (or (zero? s0-0) (= s4-1 s0-0)))
|
||||
(flush-cache 0)
|
||||
(time-of-day-interp-colors-scratch (scratchpad-ptr rgba :offset 6160) s0-0 (-> s1-0 mood))
|
||||
;; remember the previous colors
|
||||
(set! s4-1 s0-0)
|
||||
```
|
||||
where `s0-0` is the `time-of-day-pal` for the tfrag tree. It will skip interpolation if it is the same color palette that was just interpolated.
|
||||
|
||||
The `time-of-day-interp-colors-scratch` function uploads the colors from `s0-0` to the scratchpad at offset `6160`. It computes the correct colors for the time-of-day/lighting settings in the the level `s1-0`'s mood. This function is pretty complicated, so I used MIPS2C.
|
||||
|
||||
### Time of Day Interp Colors Scratch
|
||||
The very first attempts for TFRAG just skipped this function because it wasn't needed to debug the basic drawing functions. I manually set the lighting to `0.5` for all colors and `1.0` for alpha. I suspected that this stored the colors in the scratchpad. I assumed it would be fine if these garbage for a first test.
|
||||
|
||||
I noticed that this function does a few tricky things. It uses the scratchpad and it uses DMA. I know it uses DMA because I saw:
|
||||
```cpp
|
||||
c->lui(t0, 4096); // lui t0, 4096
|
||||
// some stuff in between...
|
||||
c->ori(a1, t0, 54272); // ori a1, t0, 54272 = (0x1000D400) SPR TO
|
||||
```
|
||||
and this `0x1000D4000` is the address of the DMA control register for transferring to the scratchpad. The scratchpad here is just used as a faster memory. And eventually the draw code will read the result from the scratchpad.
|
||||
|
||||
They really like this pattern of doing work on the scratchpad while DMA is running in the background, copying things to/from the scratchpad. In this case, they upload the palette to the scratchpad in chunks. As those uploads are running, they do math on the previous upload to blend together the colors for the chosen time of day. To get optimal performance, they often count how many times they finish before the DMA is ready. When this happens, they increment a "wait" variable.
|
||||
|
||||
I modified scratchpad access like this:
|
||||
```cpp
|
||||
c->lui(v1, 28672); // lui v1, 28672 0x7000
|
||||
// stuff in between skipped...
|
||||
//c->ori(v1, v1, 2064); // ori v1, v1, 2064 SPAD mods
|
||||
get_fake_spad_addr(v1, cache.fake_scratchpad_data, 2064, c);
|
||||
```
|
||||
the original code would set the address to an offset of 2064 in the scratchpad.
|
||||
|
||||
The first thing they do is wait for any in-progress DMA transfers to finish:
|
||||
```cpp
|
||||
block_1:
|
||||
c->lw(t5, 0, a1); // lw t5, 0(a1)
|
||||
// nop // sll r0, r0, 0
|
||||
// nop // sll r0, r0, 0
|
||||
// nop // sll r0, r0, 0
|
||||
c->andi(t5, t5, 256); // andi t5, t5, 256
|
||||
// nop // sll r0, r0, 0
|
||||
bc = c->sgpr64(t5) != 0; // bne t5, r0, L62
|
||||
// nop // sll r0, r0, 0
|
||||
if (bc) {goto block_1;} // branch non-likely
|
||||
```
|
||||
which is reading and checking the DMA register in a loop. We can just get rid of this - we make all DMA instant.
|
||||
|
||||
|
||||
I also modified the code that starts the transfer to just do a memcpy from the fake scratchpad. See the EE manual for details on what these registers mean. The `a1` register points to the control register for SPR TO DMA.
|
||||
```cpp
|
||||
{
|
||||
// c->sw(t4, 16, a1); // sw t4, 16(a1)
|
||||
u32 madr = c->sgpr64(t4);
|
||||
c->daddiu(t3, t3, -32); // daddiu t3, t3, -32
|
||||
// c->sw(v1, 128, a1); // sw v1, 128(a1)
|
||||
u32 sadr = c->sgpr64(v1);
|
||||
c->addiu(t5, r0, 64); // addiu t5, r0, 64
|
||||
//c->sw(t5, 32, a1); // sw t5, 32(a1)
|
||||
u32 qwc = c->sgpr64(t5);
|
||||
c->addiu(t5, r0, 256); // addiu t5, r0, 256
|
||||
// c->sw(t5, 0, a1); // sw t5, 0(a1)
|
||||
spad_to_dma(cache.fake_scratchpad_data, madr, sadr, qwc);
|
||||
c->daddiu(t4, t4, 1024); // daddiu t4, t4, 1024
|
||||
}
|
||||
```
|
||||
This data is double buffered. One buffer is being filled from DMA while another is being processed. To swap buffers, they often use `xor` to toggle a bit. But this trick only works if our buffer has the same alignment as theirs up to the bit being toggled (otherwise their first `xor` might toggle a 0 to a 1, advancing the address, where ours does the opposite).
|
||||
```
|
||||
c->xori(v1, v1, 1024); // xori v1, v1, 1024
|
||||
```
|
||||
|
||||
The actual processing is an annoying pipelined loop. The palette stores groups of 8 colors. The time of day system computes 8 weights, passed to this function. Each of the 8 colors is multiplied by the weight and added. This process is repeated for each group. There are usually 1024 or 2048 groups. The tfragments are lit by indexing into these groups. One important detail is that the r/g/b/a values are saturated so they don't overflow.
|
||||
|
||||
This part works using the MIPS2C function in the first tfrag renderers. In the third one, it was annoying to get this data to the C++ renderer, so I just recomputed it in C++. This also lets us manually override the time of day values for fun. The code is much simpler:
|
||||
```cpp
|
||||
void Tfrag3::interp_time_of_day_slow(const float weights[8],
|
||||
const std::vector<tfrag3::TimeOfDayColor>& in,
|
||||
math::Vector<u8, 4>* out) {
|
||||
for (size_t color = 0; color < in.size(); color++) {
|
||||
math::Vector4f result = math::Vector4f::zero();
|
||||
for (int component = 0; component < 8; component++) {
|
||||
result += in[color].rgba[component].cast<float>() * weights[component];
|
||||
}
|
||||
result[0] = std::min(result[0], 255.f);
|
||||
result[1] = std::min(result[1], 255.f);
|
||||
result[2] = std::min(result[2], 255.f);
|
||||
result[3] = std::min(result[3], 128.f); // note: different for alpha!
|
||||
out[color] = result.cast<u8>();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### The Call to Draw
|
||||
There was another scratchpad use to patch up here. They often treat the scratchpad as a `terrain-context`. There are quite a few overlays here so sometimes you have to do some manual searching to figure it out.
|
||||
```lisp
|
||||
(set! (-> (scratchpad-object terrain-context) bsp lev-index) (-> s1-0 index))
|
||||
|
||||
(set! (-> *tfrag-work* min-dist z) 4095996000.0)
|
||||
;; draw!
|
||||
(draw-drawable-tree-tfrag s2-0 s1-0)
|
||||
)
|
||||
;; remember closest.
|
||||
(set! (-> *level* level (-> (scratchpad-object terrain-context) bsp lev-index) closest-object 0)
|
||||
(-> *tfrag-work* min-dist z)
|
||||
)
|
||||
)
|
||||
```
|
||||
the remembering closest is used for figuring out which mip levels of texture need uploading.
|
||||
|
||||
|
||||
### Draw node culling
|
||||
This is a part that I left out. I still haven't done it. But I suspect it looks at the position of the camera (stored in `vf` regs from earlier) and modifies the visibility data. I think it uses a "sphere in view frustum" check and traverses the tree of `draw-node`s. I think it only culls the `draw-node`s and not actually the `tfragment`s, and it modifies the visibility data in place. It only culls the range of nodes that correspond to the tree we're drawing.
|
||||
|
||||
Later, on tfrag3, I did the culling in C++. (More on this later - it's done in a tricky way so that you can efficiently build a list of only the visible things to send to the GPU).
|
||||
|
||||
|
||||
### DMA List Generation
|
||||
The objective of the draw function is to generate a DMA list. This gets added to the entire DMA list for the frame and gets sent to the VIF. The DMA data is a list of instructions like:
|
||||
- upload this data to the VU memory
|
||||
- run this VU program
|
||||
- change various settings related to the VU data upload.
|
||||
|
||||
The pattern used by tfrag is:
|
||||
- Call `tfrag-init-buffer` once. This is unoptimized code that just sets things up.
|
||||
- Call `draw-inline-array-tfrag`. This adds DMA per tfragment. It is super optimized.
|
||||
- Call `tfrag-end-buffer`. This is unoptimized code that ends the DMA list for tfrag
|
||||
@ -1,227 +0,0 @@
|
||||
# Porting to x86
|
||||
|
||||
This document will keep track of stuff that needs to be ported or modified significantly for x86. Anything that uses PS2-specific hardware or relies on stuff in the C Kernel will need to be ported.
|
||||
|
||||
## Basic Info
|
||||
|
||||
Most of the game is written in GOAL. All this source lives in `goal_src`.
|
||||
|
||||
The "runtime" is all the support code written in C++. It's located in `game/`. Sometimes the "runtime" + GOAL code together is called the "target".
|
||||
|
||||
Most of the code in "runtime" is reverse engineered code from the real game, with small tweaks to make it work on x86 and with OpenGOAL.
|
||||
|
||||
The code in `game/system` is **not** from the game and is an implementation of system functions that are implemented by Sony in the PS2 game. It's stuff like threading, I/O, etc.
|
||||
|
||||
The code in `game/sce` is my implementation of the Sony libraries. When possible, I tried to keep exactly the same names/functions as the real Sony libraries. This way our reverse engineered game code can look very similar to the original, which is satisfying and fun.
|
||||
|
||||
The PS2's main CPU is called the EE. It runs GOAL code and the C Kernel, which is Naughty Dog's C++ code. The C Kernel is responsible for bootstrapping GOAL's kernel and exposing Sony library functions to GOAL code. The C Kernel is in `game/kernel`. In OpenGOAL the "EE Thread" runs code than ran on the EE on the PS2. This includes the C Kernel and all GOAL code. There is a single EE thread - GOAL implements its own threading, but this all runs in the same Linux/Windows thread.
|
||||
|
||||
The PS2 has a separate I/O Processor called the IOP. It runs the OVERLORD driver written by Naughty Dog in C. OpenGOAL uses C++ for its implementation of OVERLORD. Like with the EE, there are Sony libraries for the IOP. These are in `game/sce/iop` to distinguish them from EE code. The IOP can run independently from the EE. Unlike the EE, the IOP itself has multiple threads (7 threads) that each have their own OS thread. But only one IOP thread runs at a time, as the IOP was a single-core CPU.
|
||||
|
||||
To give an idea of the size of these (counted by `wc -l`):
|
||||
- OVERLORD is 3700 lines, but still has a lot to implement,
|
||||
- C Kernel is 7432 lines, and is mostly done
|
||||
- SCE is 973 lines, and still has some to implement
|
||||
- System is 1294 lines, and still has some to implement
|
||||
|
||||
## Math Libraries
|
||||
|
||||
I think most of the math libraries can be decompiled, but there are a few that will need manual work. These are also a great place to do tests as the math functions have very few dependencies and we know what the right answer should be for many of them.
|
||||
|
||||
- `bounding-box` (only some stuff)
|
||||
- `math` (only `rand-uint31-gen`)
|
||||
- `matrix` (only some stuff)
|
||||
- `geometry` (only some stuff)
|
||||
- `trigonometry` (only some stuff)
|
||||
|
||||
At some point we may want to re-implement some of these to be more efficient.
|
||||
|
||||
## The IOP (I/O Processor) Framework
|
||||
|
||||
This is already implemented.
|
||||
|
||||
The IOP was a separate I/O Processor on the PS2. It runs a cooperative multi-tasking kernel developed by Sony. In OpenGOAL it is implemented in `game/system/IOP_Kernel.h`. The IOP Kernel is managed by the IOP runtime thread (`game/system/iop_thread.h`).
|
||||
|
||||
The library in `game/sce/iop.h` wraps the `IOP_Kernel` in an interface that looks like the Sony libraries used by the game so the IOP code can be ported directly.
|
||||
|
||||
There are a few stub functions that are hardcoded to return the correct values for stuff like CD drive initialization. The main features currently supported are:
|
||||
- Threads (create, wakeup, start, sleep, delay, get ID)
|
||||
- Messageboxes to pass data between threads (send, poll)
|
||||
- SIF RPC, a library to receive remote procedure calls from the EE. See `game/sce/sif_ee.h` for the wrapper of the EE side library, and `game/kernel/kdgo.cpp` and `ksound.cpp` for the wrapper around the EE library that's exposed to GOAL.
|
||||
- DMA to the EE for sending data from the IOP to GOAL.
|
||||
|
||||
All this stuff is currently used for loading DGOs, which is tested and working.
|
||||
|
||||
## OVERLORD Framework
|
||||
|
||||
This is already implemented.
|
||||
|
||||
The OVERLORD is the code written by Naughty Dog that runs on the IOP. It is responsible for sound and loading data. It's surprisingly complicated and some parts of it are extremely poorly written, especially the thread synchronization stuff. My implementation of OVERLORD is in `game/overlord`. It's not complete yet, but the basics are there and it does enough to load DGOs.
|
||||
|
||||
The framework for OVERLORD is already implemented. The C Kernel calls a Sony library function to load OVERLORD. This library function is `sceSifLoadModule`, implemented in `sif_ee.cpp`, which tells the IOP Kernel code to start. This starts up the OVERLORD thread which eventually calls `start_overlord` in `game/overlord/overlord.cpp`. This `start_overlord` function is the entry point to Naughty Dog's OVERLORD library and starts a bunch more threads (see `InitISOFS`), using the Sony IOP library. In total there are 7 threads.
|
||||
|
||||
Once `start_overlord` returns, the initial call to `sceSifLoadModule` returns and the runtime keeps initializing.
|
||||
|
||||
## OVERLORD ISO Thread
|
||||
|
||||
This is partially implemented.
|
||||
|
||||
This thread is responsible for controlling the DVD drive and the small DVD data buffers used in the IOP. It has a big loop in `ISOThread()` in `iso.cpp` that looks for pending reads, executes them, waits for data to be read, then calls a callback. This code is unbelievably confusing.
|
||||
|
||||
It receives commands from other OVERLORD threads (using a MessageBox) and uses the priority queue implemented in `iso_queue.cpp` to decide which read gets to go first.
|
||||
|
||||
To interact with the DVD drive, it uses an `IsoFS` abstraction, which is a struct containing function pointers to control the drive. The version of OVERLORD in the retail game has only one implemented, called `iso_cd` which uses the actual drive in the PS2. There's also a reference to `fakeiso`, but this is empty in the game. Instead of "emulating" the CD drive functions, I implemented my own version of `fakeiso` mode in `fake_iso.cpp`. This just reads files from your hard drive and uses the `fakeiso.txt` file to map files in the `jak-project` folder to OVERLORD file names (it has it's own system for naming files).
|
||||
|
||||
It also has some sound stuff in it for managing VAG audio streams, but this isn't implemented yet.
|
||||
|
||||
The other threads in OVERLORD are "RPC" threads. They sit in a loop waiting for the main runtime thread (EE thread) to send a remote procedure call (RPC). Then they do something (like maybe sending a message to the ISO thread), maybe wait for something to happen, and then return.
|
||||
|
||||
From the GOAL/EE side of things, RPC calls can be blocking or non-blocking. They can be issued from GOAL (with `rpc-call`) or from the C Kernel (`RpcCall`). Per "channel" (corresponds to an IOP thread), there can only be one RPC call happening at a time. The `rpc-busy?` command can be used to check if an RPC is complete.
|
||||
|
||||
## IOP PLAY (6)
|
||||
|
||||
This is unimplemented.
|
||||
|
||||
The `PLAY` RPC appears to be relatively simple and plays/stops/pauses/queues a VAG audio stream. It can either use the "AnimationName" system or another system to get the name of the audio stream. I don't know what sound effects in the game are streamed, but I believe there are some.
|
||||
|
||||
I suspect the GOAL side code for this is in `gsound` and `gsound-h`.
|
||||
|
||||
## IOP STR (5)
|
||||
|
||||
This is unimplemented.
|
||||
|
||||
This is an RPC for streaming data back to the EE. I think this is used to control animation streaming.
|
||||
|
||||
## IOP DGO (4)
|
||||
|
||||
This is implemented.
|
||||
|
||||
This is the RPC for loading DGO files. The DGO loading is super complicated, but the basic idea is that loading / linking are double buffered. In order to allow linking files to allocate memory, the currently loading file goes in a temporary buffer on the top of the heap. (There are actually two temp buffers that rotate, one for loading out of and one for linking, as the "copy to heap" step is done as part of linking, not loading)
|
||||
|
||||
The final chunk is not double buffered. This is so it can be loaded directly into its final location in the heap. This has three advantages: you don't need to copy it out of a temporary buffer, you can have a file larger than the temp buffer and you can also entirely fill the heap this way (the temp buffers are freed so you don't have to worry about that).
|
||||
|
||||
The IOP side state machine for this is in `iso.cpp`, implemented inside of the DGO load buffer complete callback and is somewhat complicated because DGO info may be split between multiple buffers, and you have to deal with getting partial info. The EE side is in `kdgo.cpp`.
|
||||
|
||||
The DGO synchronization is pretty confusing but I believe I have it working. It may be worth documenting it more (I thought I did already, but where did I put it?).
|
||||
|
||||
## IOP Server/Ramdisk (3)
|
||||
|
||||
This is implemented, but so far unused and untested.
|
||||
|
||||
This RPC is used to store files in RAM on the IOP. There's a buffer of around 800 kB. I believe it's used for a few different things, in particular the level visibility data. The EE requests data to be loaded from a file on DVD into the "ramdisk" (just a buffer on the IOP), then can request chunks of this file. Of course it is not as fast as storing the file in the EE RAM, but it is much faster than reading from the DVD again.
|
||||
|
||||
This is what Andy Gavin refers to when they said they did "things they weren't supposed to" with the "one megabyte of memory that wasn't being used".
|
||||
|
||||
|
||||
## IOP Loader (2)
|
||||
|
||||
This is unimplemented.
|
||||
|
||||
This is used to control the loading of music and soundbanks. I haven't touched it yet. Music and soundbanks are loaded into IOP memory when you switch levels.
|
||||
|
||||
## IOP Player (1)
|
||||
|
||||
This is unimplemented.
|
||||
|
||||
This is used to control the playing of sound, and goes with Loader. Like PLAY it can play VAG audio streams. I'm not sure which one is actually used for streaming audio, maybe both?
|
||||
|
||||
## IOP VBlank Handler
|
||||
|
||||
This is unimplemented.
|
||||
|
||||
The IOP's kernel will call `VBlank_Handler` on each vblank. This is once per frame, and I don't know where it is, or if its tied to the actual HW vblank or framebuffer swap, if it happens at 30/60 fps (or even/odd frames if 30 fps). I suspect it's the real vblank at 60 fps but I don't know.
|
||||
|
||||
This does some music fade calculations and sends some data to the EE. In GOAL this is called the `sound-iop-info`.
|
||||
|
||||
The EE first has to do some set up to tell the IOP where to copy the data, which I believe is done in another sound RPC from GOAL.
|
||||
|
||||
We'll also need to add some stuff to `system` and `sce/iop` to set this up, which will have to work with frame timing stuff so it happens at the right part of the frame.
|
||||
|
||||
## Sound Library
|
||||
|
||||
This is a pretty big one. To actually make sounds, OVERLORD code uses a third-party sound library called 989SND. Internally 989SND uses the SPU2 (Sound Processor) to actually do the "sound math" to decode ADPCM, do ADSR for the sampled sounds, and do reverb/mixing.
|
||||
|
||||
I think the lowest effort sound implementation is to try to reimplement 989SND + the SPU as a single library. This could be tested and developed in isolation from everything else.
|
||||
|
||||
We'll also need to pick a library for doing audio on PC and a design for how to keep the audio in sync. My gut feeling is to let the IOP side audio stuff just run totally independent from everything else, like the real game does. Let the audio sampling be driven by the sound device so you never have any crackling/interpolation artifacts. This is why the audio keeps going even after the game crashes on PS2.
|
||||
|
||||
## GOAL Kernel
|
||||
|
||||
The GOAL kernel needs some modification to work on x86. It implements userspace threading and needs to know the details of how to back up the current CPU state and restore it. It also needs to work with the compiler to make sure that the kernel and compiler agree on what registers may not be preserved across a thread suspend There are also some CPU specific details on how to do dynamic throw/catches, unwinding stack frames, and passing initial arguments to a thread.
|
||||
|
||||
In OpenGOAL, the `rsp` is a "real" pointer and all other pointers are "GOAL pointer"s (offset from base of GOAL memory), so there are some details needed to correctly save/restore stacks.
|
||||
|
||||
A final detail is we will probably want/need the ability to increase the default size of stack that can be backed up on suspend. The default is 256 bytes so if our compiler does worse than the original and we use more stack space, we could run out. There's a check for this so it shouldn't be hard to detect.
|
||||
|
||||
## Jak Graphics Basics
|
||||
|
||||
The PS2 has some special hardware that's used for graphics. These are the DMAC, the VU1, and the GS.
|
||||
|
||||
The DMAC is a sophisticated DMA controller. It runs separately from the EE and can copy data from one place to another at pretty high speed. If it is not stalled for any reason it can reach 2.4 GB/sec. The main RAM is only good for around 1.2 GB/sec so in practice "big" things don't move around any faster than 1.2 GB/sec on average. It's used to send graphics data from main memory to the other components. It can be configured, but it's not programmable. It can do simple transfers, like "copy this block of data from here to there", and more complicated things, like following linked lists.
|
||||
|
||||
The VU1 takes the role of vertex shaders. It can be programmed, but only in assembly, and it is extremely challenging and confusing. It has an extremely small memory (16 kB), but this memory is extremely fast. It's role is usually to do vertex transformations and lighting, then generate a list of commands to send to the GS. The `XGKICK` instruction on VU1 is used to send data from the VU1 memory to the GS.
|
||||
|
||||
The GS is the actual GPU. It has VRAM and receives commands from a few different places, including:
|
||||
- VU1 `XGKICK`s stuff to it directly, bypassing the main bus used by DMAC/CPU memory access. This is called PATH 1 and is most commonly used in Jak 1.
|
||||
- When DMAing stuff to VU1, it first goes through a thing called VIF1 which can "unpack" data. There is a special command that you can give to VIF1 which tells it to "send data directly to the GS".
|
||||
- DMA sends data directly from EE main memory to GS (Path 3), unused by Jak 1
|
||||
|
||||
The GS is like pixel shaders but it's very simple - it's not programmable and only can do a few fixed things. The GS also has the VRAM, which can contain frame buffers, z buffers, textures, and scratch area for effects.
|
||||
|
||||
My understanding is that during a frame, the EE generates a long list of things to draw. These are a DMA "chain" - basically a complicated linked-list like data structure that the PS2's DMA knows how to handle. I believe some graphics calculations are done on the EE - particularly the environment mapping.
|
||||
|
||||
## DMA
|
||||
|
||||
## Display
|
||||
|
||||
## Texture
|
||||
|
||||
## Collision System
|
||||
|
||||
## Joint
|
||||
|
||||
## BSP
|
||||
|
||||
## Merc Blend Shape
|
||||
|
||||
## Ripple
|
||||
|
||||
## Bones
|
||||
|
||||
## Generic Merc
|
||||
|
||||
## Generic TIE
|
||||
|
||||
## Shadow
|
||||
|
||||
## Font
|
||||
|
||||
## Decompression
|
||||
|
||||
## Background
|
||||
|
||||
## Draw Node Culling
|
||||
|
||||
## Shrubbery
|
||||
|
||||
## TFRAG
|
||||
|
||||
## TIE
|
||||
|
||||
## Particle
|
||||
|
||||
## Time of Day
|
||||
|
||||
## Sky
|
||||
|
||||
## Load boundary
|
||||
|
||||
## Sound
|
||||
|
||||
## Controllers
|
||||
|
||||
## IOP Streaming
|
||||
|
||||
## Ocean
|
||||
|
||||
## Navigate
|
||||
@ -1,175 +0,0 @@
|
||||
# Process and State
|
||||
|
||||
## What is a `process`?
|
||||
|
||||
A `process` object stores the state of some in-game object and tells the GOAL kernel how to update this object on each frame.
|
||||
|
||||
For example, there is a process for Jak, a process for each orb, and a process for each enemy. There is also a process for the time-of-day system and the pause menu.
|
||||
|
||||
In most cases, `process` is used as a parent type for a specific game object. For example, `money` (orb) is a child of `process-drawable`, which is a child of `process`. A `process-drawable` is a process that can be drawn as part of the `drawable` system.
|
||||
|
||||
## What does a process store?
|
||||
|
||||
Each `process` stores a small amount (112 bytes) of metadata, fields from child classes, some unknown stuff, and a process heap. The process heap will automatically contain the "main thread" of the process, which contains space to back up the stack and registers when the thread suspends. You may also allocate objects on the process heap yourself (not supported in OpenGOAL yet).
|
||||
|
||||
## How is a process run?
|
||||
|
||||
The `process` class is a child of `process-tree`, which is a left-child right-sibling binary tree. On each frame, the kernel iterates through the `*active-pool*` and runs each process. Each run consists of three steps:
|
||||
|
||||
- Run the `trans-hook` of the process in a temporary stack.
|
||||
- Resume the main thread of the process.
|
||||
- After the main thread suspends, run the `post-hook`.
|
||||
|
||||
|
||||
## How do I create a process?
|
||||
|
||||
Setting up a process requires three steps:
|
||||
- Getting an actual process object
|
||||
- "Activating" a process so it will be run by the kernel
|
||||
- Setting up the code for the process to run
|
||||
|
||||
|
||||
There are a few "dead pools" which contain process objects that are not in use. The `*4k-dead-pool*` contains processes that are 4kb each. There is also a dynamic pool called the `*nk-dead-pool*` that allows you to create dynamically sized processes. You must do all allocations during initialization with these processes because they automatically "shrink" their heap as small as possible. Also, `*nk-dead-pool*` processes will be relocated in memory as part of the process GC system, so you must make sure that all objects on the process heap support relocation, and you must use a `handle` to safely refer to the process, not just a normal `process` reference.
|
||||
|
||||
For example, to get a process:
|
||||
```
|
||||
gc> (define *test-proc* (get-process *nk-dead-pool* process 1024))
|
||||
#<process process dead :state #f :stack -1904/1441188 :heap 0/1024 @ #x193454>
|
||||
```
|
||||
|
||||
This shows that:
|
||||
- The process name is `process` (just a temporary name, until we activate)
|
||||
- The status is `dead`
|
||||
- The process is not in a `state`.
|
||||
- The stack is bogus because we don't have a main thread yet.
|
||||
- We have used 0 out of 1024 bytes of our process heap.
|
||||
|
||||
Next, we need to activate it:
|
||||
```
|
||||
(activate *test-proc* *active-pool* 'hello *kernel-dram-stack*)
|
||||
```
|
||||
|
||||
This means:
|
||||
- We put it in the `*active-pool*`. We could specify another process in the `*active-pool*` if we wanted this to be a child process of an existing process.
|
||||
- Our name is `'hello`.
|
||||
- When we run code, it will run on the `*kernel-dram-stack*`.
|
||||
|
||||
Now, if we `(print *test-proc*)` we will see:
|
||||
```
|
||||
#<process hello ready :state #f :stack 0/256 :heap 384/1024 @ #x193454>
|
||||
```
|
||||
Indicating that we are "ready" to be initialized, and that we now have a correctly set up main thread/stack.
|
||||
|
||||
If we run `inspect`, it will print out all objects on the process heap, including our main thread:
|
||||
```
|
||||
----
|
||||
[001934c4] cpu-thread
|
||||
name: code
|
||||
process: #<process hello ready :state #f :stack 0/256 :heap 384/1024 @ #x193454>
|
||||
previous: #f
|
||||
suspend-hook: #<compiled function @ #x1679c4>
|
||||
resume-hook: #<compiled function @ #x167b24>
|
||||
pc: #x0
|
||||
sp: #x170b30
|
||||
stack-top: #x170b30
|
||||
stack-size: 256
|
||||
rreg[7] @ #x1934e8
|
||||
freg[8] @ #x193520
|
||||
stack[0] @ #x193540
|
||||
----
|
||||
```
|
||||
|
||||
If we want a reference to this process, we must create a handle. For example:
|
||||
```
|
||||
gc> (process->handle *test-proc*)
|
||||
#<handle :process #<process hello ready :state #f :stack 0/256 :heap 384/1024 @ #x192fe4> :pid 2>
|
||||
```
|
||||
this is now a safe reference to this process, even if it is relocated or deactivated.
|
||||
|
||||
## How do I make a process do something?
|
||||
The `state` system is used to control a process. Each process can be in a `state`, which specifies what functions should run. To switch states in the current process, use `go`.
|
||||
|
||||
For example, we can create a simple test state like this:
|
||||
```
|
||||
(defstate test-state (process)
|
||||
:enter (lambda () (format #t "enter!~%"))
|
||||
:exit (lambda () (format #t "exit!~%"))
|
||||
:trans (lambda () (format #t "trans!~%"))
|
||||
:post (lambda () (format #t "post!~%"))
|
||||
:code (lambda ()
|
||||
(dotimes (i 5)
|
||||
(format #t "Code ~D~%" i)
|
||||
(suspend)
|
||||
)
|
||||
(process-deactivate)
|
||||
)
|
||||
)
|
||||
```
|
||||
The `code` is the function to run in the main thread. This code should `suspend` itself, and the kernel will resume it after the suspend on each frame. Once the process is done, it can call `process-deactivate`. This will cause it to exit the current state, immediately exit the `code`, and clean up the process, returning it to the dead pool.
|
||||
|
||||
To switch the process to this state, you can use the `run-now-in-process` to switch to the test process and run the given code.
|
||||
```
|
||||
(run-now-in-process *test-proc* (lambda () (go test-state)))
|
||||
```
|
||||
|
||||
And you will see:
|
||||
```
|
||||
enter!
|
||||
trans!
|
||||
Code 0
|
||||
post!
|
||||
|
||||
trans!
|
||||
Code 1
|
||||
post!
|
||||
|
||||
trans!
|
||||
Code 2
|
||||
post!
|
||||
|
||||
trans!
|
||||
exit!
|
||||
```
|
||||
|
||||
Note 1: After deactivation, the handle is no longer valid as the process is dead and it will print like this:
|
||||
```
|
||||
#<handle :process #f :pid 2>
|
||||
```
|
||||
|
||||
Note 2: There is also a `run-next-time-in-process` that sets up the process to run your initialization stub function as the `code` on the next time the kernel iterates through the process tree.
|
||||
|
||||
## Some notes on "the current process"
|
||||
When the kernel runs a process, it sets `(-> *kernel-context* current-process)` and the `pp` register to that process. This process is called the "current kernel process".
|
||||
|
||||
This process may then "run code in another process". This can be done with `run-now-in-process`, by deactivating another process, or using `go` on another process. This changes `pp`, but not the kernel context. The process in `pp` is called the "current pp process".
|
||||
|
||||
The value of the `pp` register determines the current process.
|
||||
|
||||
|
||||
## Some notes on `process-deactivate`
|
||||
To stop a process, you can do call the `deactivate` method of that process. The `process-deactivate` macro just does this for the current process.
|
||||
|
||||
This does the following:
|
||||
- Set state to `dead-state`.
|
||||
- Calls `entity-deactivate-handler`, if you have an entity
|
||||
- Calls `exit` of states
|
||||
- Cleans up any pending `protect-frame` (calling them with pp set for the process)
|
||||
- Disconnects it from the `connection` system
|
||||
- Deactivates all children process
|
||||
- Returns itself to the pool
|
||||
- If you deactivated the process that the kernel-dispatcher started running, immediately bail out of the thread
|
||||
- If you deactivated during a `run-now-in-process`, immediately bail out of the initialization and return to caller of `run-now-in-process`.
|
||||
|
||||
## Some notes on `go`.
|
||||
The `go` macro is used to change the state of the current process.
|
||||
|
||||
|
||||
If you use `go` when in `run-now-in-process`, it will immediately return to the caller of `run-now-in-process`, and the actual state change will happen on the next execution of the main thread of that process.
|
||||
|
||||
If you use `go-process` on another process, the `go-process` will return immediately and the state transition will happen on the next run of that process.
|
||||
|
||||
If you use `go` in the main thread, it will immediately transition states, run exits, enter, trans, and begin running the new state `code`.
|
||||
|
||||
If you use `go` in `trans` it will set up the next run of the main thread, then abandon the current `trans`.
|
||||
If you use `go` in `post`, it will set up the next run of the main thread to transition, but not abandon the current `post`.
|
||||
|
||||
@ -1,96 +0,0 @@
|
||||
# Reader
|
||||
|
||||
GOOS and GOAL both use the same reader, which converts text files to S-Expressions and allows these s-expressions to be mapped back to a line in a source file for error messages. This document explains the syntax of the reader. Note that these rules do not explain the syntax of the language (for instance, GOAL has a much more complicated system of integers and many more restrictions), but rather the rules of how your program source must look.
|
||||
|
||||
## Integer Input
|
||||
|
||||
Integers handled by the reader are 64-bits. Any overflow is considered an error. An integer can be specified as a decimal, like `0` or `-12345`; in hex, like `#xbeef`; or in binary, like `#b101001`. All three representations can be used anywhere an integer is used. Hex numbers do not care about the case of the characters. Decimal numbers are signed, and wrapping from a large positive number to a negative number will generate an error. The valid input range for decimals is `INT64_MIN` to `INT64_MAX`. Hex and binary are unsigned and do not support negative signs, but allow large positive numbers to wrap to negative. Their input range is `0` to `UINT64_MAX`. For example, `-1` can be entered as `-1` or `#xffffffffffffffff`, but not as `UINT64_MAX` in decimal.
|
||||
|
||||
## Floating Point Input
|
||||
|
||||
Floating point values handled by the reader are implemented with `double`. Weird numbers (denormals, NaN, infinity) are invalid and not handled by the reader directly. A number _must_ have a decimal point to be interpreted as floating point. Otherwise, it will be an integer. Leading/trailing zeros are optional.
|
||||
|
||||
## Character Input
|
||||
|
||||
Characters are used to represent characters that are part of text. The character `c` is represented by `#\c`. This representation is used for all ASCII characters between `!` and `~`. There are three special characters which have a non-standard representation:
|
||||
- Space : `#\\s`
|
||||
- New Line: `#\\n`
|
||||
- Tab: `#\\t`
|
||||
|
||||
All other characters are invalid.
|
||||
|
||||
## Strings
|
||||
|
||||
A string is a sequence of characters, surrounding by double quotes. The ASCII characters from ` ` to `~` excluding `"` can be entered directly. Strings have the following escape codes:
|
||||
- `\\` : insert a backslash
|
||||
- `\n` : insert a new line
|
||||
- `\t` : insert a tab
|
||||
- `\"` : insert a double quote
|
||||
|
||||
## Comments
|
||||
|
||||
The reader supports line comments with `;` and multi-line comments with `#| |#`. For example
|
||||
|
||||
```lisp
|
||||
(print "hi") ; prints hi
|
||||
|
||||
#|
|
||||
this is a multi-line comment!
|
||||
(print "hi") <- this is commented out.
|
||||
|#
|
||||
```
|
||||
|
||||
## Array
|
||||
|
||||
The reader supports arrays with the following syntax:
|
||||
```
|
||||
; array of 1, 2, 3, 4
|
||||
#(1 2 3 4)
|
||||
```
|
||||
|
||||
Arrays can be nested with lists, pairs, and other arrays.
|
||||
|
||||
## Pair
|
||||
|
||||
The reader supports pairs with the following syntax:
|
||||
|
||||
```lisp
|
||||
; pair of a, b
|
||||
(a . b)
|
||||
```
|
||||
|
||||
Pairs can be nested with lists, pairs, and arrays.
|
||||
|
||||
## List
|
||||
|
||||
The reader supports lists. Lists are just an easier way of constructing a linked list of pairs, terminated with the empty list. The empty list is a special list written like `()`.
|
||||
|
||||
```lisp
|
||||
; list of 1, 2, 3
|
||||
(1 2 3)
|
||||
; actually the same as
|
||||
(1 . (2 . (3 . ())))
|
||||
```
|
||||
|
||||
## Symbol
|
||||
|
||||
A symbol is a sequence of characters containing no whitespace, and not matching any other data type. (Note: this is not a very good definition). Typically symbols are lower case, and words are separated by a `-`. Examples:
|
||||
|
||||
```lisp
|
||||
this-is-a-symbol
|
||||
; you can have weird symbols too:
|
||||
#f
|
||||
#t
|
||||
-
|
||||
*
|
||||
+
|
||||
__WEIRDLY-NamedSymbol ; this is weird, but OK.
|
||||
```
|
||||
|
||||
## Reader Macros
|
||||
|
||||
The reader has some default macros which are common in Scheme/LISP:
|
||||
- `'x` will be replaced with `(quote x)`
|
||||
- `` `x`` will be replaced with `(quasiquote x)`
|
||||
- `,x` will be replaced with `(unquote x)`
|
||||
- `,@` will be replaced with `(unquote-splicing x)`
|
||||
@ -1,128 +0,0 @@
|
||||
# Registers
|
||||
|
||||
Although modern computers are much faster than the PS2, and we could probably get away with a really inefficient register allocation scheme, I think it's worth it to get this right.
|
||||
|
||||
## Register differences between MIPS and x86-64
|
||||
|
||||
The PS2's MIPS processor has these categories of register:
|
||||
- General Purpose. They are 128-bit, but usually only lower 64 bits are used. 32 registers, each 128-bits.
|
||||
- Floating point registers. 32 registers, each for a 32-bit float.
|
||||
- Vector float registers. 32 registers, each for 4x 32-bit floats. Used only in inline assembly
|
||||
- `vi` registers. 16 registers, each a 16-bit integer. Used very rarely in inline assembly
|
||||
|
||||
There are also some control/special registers too (`Q`, `R`...), but code using these will be manually ported.
|
||||
|
||||
In comparison, x86-64 has much fewer registers:
|
||||
- 16 General Purpose. Each 64-bits
|
||||
- 16 `xmm` registers. 128-bits, and can store either 128-bit integers or 4x 32-bit floats
|
||||
|
||||
Here is the mapping:
|
||||
- MIPS GPR (lower 64 bits only) - x86-64 GPR
|
||||
- MIPS GPR (128-bits, only special cases) - x64-64 `xmm`
|
||||
- MIPS floating point - x64-64 `xmm` (lower 32-bits)
|
||||
- MIPS vector float - x64-64 `xmm` (packed single)
|
||||
- MIPS `vi` - manually handled??
|
||||
|
||||
Here is the MIPS GPR map
|
||||
- `r0` or `zero` : always zero
|
||||
- `r1` or `at`: assembler temporary, not saved, not used by compiler
|
||||
- `r2` or `v0`: return value, not saved
|
||||
- `r3` or `v1`: not saved
|
||||
- `r4` or `a0`: not saved, argument 0
|
||||
- `r5` or `a1`: not saved, argument 1
|
||||
- `r6` or `a2`: not saved, argument 2
|
||||
- `r7` or `a3`: not saved, argument 3
|
||||
- `r8` or `t0`: not saved, argument 4
|
||||
- `r9` or `t1`: not saved, argument 5
|
||||
- `r10` or `t2`: not saved, argument 6
|
||||
- `r11` or `t3`: not saved, argument 7
|
||||
- `r12` or `t4`: not saved
|
||||
- `r13` or `t5`: not saved
|
||||
- `r14` or `t6`: not saved
|
||||
- `r15` or `t7`: not saved
|
||||
- `r16` or `s0`: saved
|
||||
- `r17` or `s1`: saved
|
||||
- `r18` or `s2`: saved
|
||||
- `r19` or `s3`: saved
|
||||
- `r20` or `s4`: saved
|
||||
- `r21` or `s5`: saved
|
||||
- `r22` or `s6`: saved, process pointer
|
||||
- `r23` or `s7`: saved, symbol pointer
|
||||
- `r24` or `t8`: not saved
|
||||
- `r25` or `t9`: function call pointer
|
||||
- `r26` or `k0`: kernel reserved (unused)
|
||||
- `r27` or `k1`: kernel reserved (unused)
|
||||
- `r28` or `gp`: saved
|
||||
- `r29` or `sp`: stack pointer
|
||||
- `r30` or `fp`: current function pointer
|
||||
- `r31` or `ra`: return address pointer
|
||||
|
||||
|
||||
And the x86-64 GPR map
|
||||
- `rax`: return value
|
||||
- `rcx`: argument 3
|
||||
- `rdx`: argument 2
|
||||
- `rbx`: saved
|
||||
- `rsp`: stack pointer
|
||||
- `rbp`: saved
|
||||
- `rsi`: argument 1
|
||||
- `rdi`: argument 0
|
||||
- `r8`: argument 4
|
||||
- `r9`: argument 5
|
||||
- `r10`: argument 6, saved if not argument
|
||||
- `r11`: argument 7, saved if not argument
|
||||
- `r12`: saved
|
||||
- `r13`: process pointer
|
||||
- `r14`: symbol table
|
||||
- `r15`: offset pointer
|
||||
|
||||
### Plan for Memory Access
|
||||
|
||||
The PS2 uses 32-bit pointers, and changing the pointer size is likely to introduce bugs, so we will keep using 32-bit pointers. Also, GOAL has some hardcoded checks on the value for pointers, so we need to make sure the memory appears to the program at the correct address.
|
||||
|
||||
To do this, we have separate "GOAL Pointers" and "real pointers". The "real pointers" are just normal x86-64 pointers, and the "GOAL Pointer" is an offset into a main memory array. A "real pointer" to the main memory array is stored in `r15` (offset pointer) when GOAL code is executing, and the GOAL compiler will automatically add this to all memory accesses.
|
||||
|
||||
The overhead from doing this is not as bad as you might expect - x86 has nice addressing modes (Scale Index Base) which are quite fast, and don't require the use of temporary registers. If this does turn out to be much slower than I expect, we can introduce the concept of real pointers in GOAL code, and use them in places where we are limited in accessing memory.
|
||||
|
||||
The main RAM is mapped at `0x0` on the PS2, with the first 1 MB reserved for the kernel. We should make sure that the first 1 MB of GOAL main memory will cause a segfault if read/written/executed, to catch null pointer bugs.
|
||||
|
||||
In the C Kernel code, the `r15` pointer doesn't exist. Instead, `g_ee_main_memory` is a global which points to the beginning of GOAL main memory. The `Ptr<T>` template class takes care of converting GOAL and C++ pointers in a convenient way, and catches null pointer access.
|
||||
|
||||
The GOAL stack pointer should likely be a real pointer, for performance reasons. This makes pushing/popping/calling/returning/accessing stack variables much faster (can use actual `push`, `pop`), with the only cost being getting a GOAL stack pointer requiring some extra work. The stack pointer's value is read/written extremely rarely (only in kernel code that will be rewritten anyway), so this seems like a good tradeoff.
|
||||
|
||||
The other registers are less clear. The process pointer can probably be a real pointer. But the symbol table could go a few ways:
|
||||
1. Make it a real pointer. Symbol value access is fast, but comparison against false requires two extra operations.
|
||||
2. Make it a GOAL pointer. Symbol value access requires more complicated addressing modes to be one instruction, but comparison against false is fast.
|
||||
|
||||
Right now I'm leaning toward 2, but it shouldn't be a huge amount of work to change if I'm wrong.
|
||||
|
||||
### Plan for Function Call and Arguments
|
||||
|
||||
In GOAL for MIPS, function calls are weird. Functions are always called by register using `t9`. There seems to be a different register allocator for function pointers, as nested function calls have really wacky register allocation. In GOAL-x86-64, this restriction will be removed, and a function can be called from any register. (see next section for why we can do this)
|
||||
|
||||
Unfortunately, GOAL's 128-bit function arguments present a big challenge. When calling a function, we can't know if the function we're calling is expecting an integer, float, or 128-bit integer. In fact, the caller may not even know if it has an integer, float, or 128-bit integer. The easy and foolproof way to get this right is to use 128-bit `xmm` registers for all arguments and return values, but this will cause a massive performance hit and increase code size, as we'll have to move values between register types constantly. The current plan is this:
|
||||
|
||||
- Floats go in GPRs for arguments/return values. GOAL does this too, and takes the hit of converting between registers as well. Probably the impact on a modern CPU is even worse, but we can live with it.
|
||||
- We'll compromise for 128-bit function calls. When the compiler can figure out that the function being called expects or returns a 128-bit value, it will use the 128-bit calling convention. In all other cases, it will use 64-bit. There aren't many places where 128-bit integer are used outside of inline assembly, so I suspect this will just work. If there are more complicated instances (call a function pointer and get either a 64 or 128-bit result), we will need to special case them.
|
||||
|
||||
### Plan for Static Data
|
||||
|
||||
The original GOAL implementation always called functions by using the `t9` register. So, on entry to a function, the `t9` register contains the address of the function. If the function needs to access static data, it will move this `fp`, then do `fp` relative addressing to load data. Example:
|
||||
|
||||
```nasm
|
||||
function-start:
|
||||
daddiu sp, sp, -16 ;; allocate space on stack
|
||||
sd fp, 8(sp) ;; back up old fp on stack
|
||||
or fp, t9, r0 ;; set fp to address of function
|
||||
lwc1 f0, L345(fp) ;; load relative to function start
|
||||
```
|
||||
|
||||
To copy this exactly on x86 would require reserving two registers equivalent to `t9` and `gp`. A better approach for x86-64 is to use "RIP relative addressing". This can be used to load memory relative to the current instruction pointer. This addressing mode can be used with "load effective address" (`lea`) to create pointers to static data as well.
|
||||
|
||||
### Plan for Memory
|
||||
|
||||
Access memory by GOAL pointer in `rx` with constant offset (optionally zero):
|
||||
|
||||
```nasm
|
||||
mov rdest, [roff + rx + offset]
|
||||
```
|
||||
@ -1,304 +0,0 @@
|
||||
# Compiler REPL
|
||||
|
||||
When you start the OpenGOAL compiler, you'll see a prompt like this:
|
||||
|
||||
```lisp
|
||||
OpenGOAL Compiler 0.2
|
||||
g >
|
||||
```
|
||||
|
||||
The `g` indicates that you can input OpenGOAL compiler commands. For a listing of common commands run:
|
||||
|
||||
```lisp
|
||||
(repl-help)
|
||||
```
|
||||
|
||||
## Connecting To Target Example
|
||||
|
||||
In order to execute OpenGOAL code, you must connect to the listener.
|
||||
|
||||
```lisp
|
||||
;; we cannot execute OpenGOAL code unless we connect the listener
|
||||
g > (+ 1 2 3)
|
||||
REPL Error: Compilation generated code, but wasn't supposed to
|
||||
|
||||
;; connect to the target
|
||||
g > (lt)
|
||||
[Listener] Socket connected established! (took 0 tries). Waiting for version...
|
||||
Got version 0.2 OK!
|
||||
[Debugger] Context: valid = true, s7 = 0x147d24, base = 0x2000000000, tid = 1297692
|
||||
|
||||
;; execute OpenGOAL code
|
||||
gc > (+ 1 2 3)
|
||||
6
|
||||
|
||||
;; quit the compiler and reset the target for next time.
|
||||
gc > (e)
|
||||
[Listener] Closed connection to target
|
||||
```
|
||||
|
||||
Once we are connected, we see that there is a `gc` prompt. This indicates that the listener has an open socket connection. Now the REPL will accept both compiler commands and OpenGOAL source code. All `(format #t ...` debugging prints (like `printf`) will show up in this REPL. Each time you run something in the REPL, the result is printed as a decimal number. If the result doesn't make sense to print as a decimal, or there is no result, you will get some random number.
|
||||
|
||||
In the future there will be a fancier printer here.
|
||||
|
||||
## General Command Listing
|
||||
|
||||
### `(e)`
|
||||
|
||||
```lisp
|
||||
(e)
|
||||
```
|
||||
|
||||
Exit the compiler once the current REPL command is finished. Takes no arguments. If we are connected to a target through the listener, attempts to reset the target.
|
||||
|
||||
### `(:exit)`
|
||||
|
||||
Exit Compiler
|
||||
|
||||
```lisp
|
||||
(:exit)
|
||||
```
|
||||
|
||||
Same as `(e)`, just requires more typing. `(e)` is actually a macro for `(:exit)`. Takes no arguments.
|
||||
|
||||
### `(lt)`
|
||||
|
||||
Listen to Target
|
||||
|
||||
```lisp
|
||||
(lt ["ip address"] [port-number])
|
||||
```
|
||||
|
||||
Connect the listener to a running target. The IP address defaults to `"127.0.0.1"` and the port to `8112` (`DECI2_PORT` in `listener_common.h`). These defaults are usually what you want, so you can just run `(lt)` to connect.
|
||||
|
||||
Example:
|
||||
|
||||
```lisp
|
||||
g > (lt)
|
||||
[Listener] Socket connected established! (took 0 tries). Waiting for version...
|
||||
Got version 0.2 OK!
|
||||
[Debugger] Context: valid = true, s7 = 0x147d24, base = 0x2000000000, tid = 1296302
|
||||
gc >
|
||||
```
|
||||
|
||||
### `r`
|
||||
|
||||
Reset the target.
|
||||
|
||||
```lisp
|
||||
(r ["ip address"] [port-number])
|
||||
```
|
||||
|
||||
Regardless of the current state, attempt to reset the target and reconnect. After this, the target will have nothing loaded. Like with `(lt)`, the default IP and port are probably what you want.
|
||||
|
||||
Note: `r` is actually a macro.
|
||||
|
||||
### `shutdown-target`
|
||||
|
||||
If the target is connected, make it exit.
|
||||
|
||||
```lisp
|
||||
(shutdown-target)
|
||||
```
|
||||
|
||||
The target will print
|
||||
|
||||
```
|
||||
GOAL Runtime Shutdown (code 2)
|
||||
```
|
||||
|
||||
before it exits.
|
||||
|
||||
### `:status`
|
||||
|
||||
Ping the target.
|
||||
|
||||
```lisp
|
||||
(:status)
|
||||
```
|
||||
|
||||
Send a ping-like message to the target. Requires the target to be connected. If successful, prints nothing. Will time-out and display and error message if the GOAL kernel or code dispatched by the kernel is stuck in an infinite loop. Unlikely to be used often.
|
||||
|
||||
## Compiler Forms - Compiler Commands
|
||||
|
||||
These forms are used to control the GOAL compiler, and are usually entered at the GOAL REPL, or as part of a macro that's executed at the GOAL REPL. These shouldn't be used in GOAL source code.
|
||||
|
||||
### `reload`
|
||||
|
||||
Reload the GOAL compiler
|
||||
```lisp
|
||||
(reload)
|
||||
```
|
||||
|
||||
Disconnect from the target and reset all compiler state. This is equivalent to exiting the compiler and opening it again.
|
||||
|
||||
### `get-info`
|
||||
|
||||
Get information about something.
|
||||
|
||||
```lisp
|
||||
(get-info <something>)
|
||||
```
|
||||
|
||||
Use `get-info` to see what something is and where it is defined.
|
||||
|
||||
For example:
|
||||
|
||||
```lisp
|
||||
;; get info about a global variable:
|
||||
g > (get-info *kernel-context*)
|
||||
[Global Variable] Type: kernel-context Defined: text from goal_src/kernel/gkernel.gc, line: 88
|
||||
(define *kernel-context* (new 'static 'kernel-context
|
||||
|
||||
;; get info about a function. This particular function is forward declared, so there's an entry for that too.
|
||||
;; global functions are also global variables, so there's a global variable entry as well.
|
||||
g > (get-info fact)
|
||||
[Forward-Declared] Name: fact Defined: text from goal_src/kernel/gcommon.gc, line: 1098
|
||||
(define-extern fact (function int int))
|
||||
|
||||
[Function] Name: fact Defined: text from kernel/gcommon.gc, line: 1099
|
||||
(defun fact ((x int))
|
||||
|
||||
[Global Variable] Type: (function int int) Defined: text from goal_src/kernel/gcommon.gc, line: 1099
|
||||
(defun fact ((x int))
|
||||
|
||||
;; get info about a type
|
||||
g > (get-info kernel-context)
|
||||
[Type] Name: kernel-context Defined: text from goal_src/kernel/gkernel-h.gc, line: 114
|
||||
(deftype kernel-context (basic)
|
||||
|
||||
;; get info about a method
|
||||
g > (get-info reset!)
|
||||
[Method] Type: collide-sticky-rider-group Method Name: reset! Defined: text from goal_src/engine/collide/collide-shape-h.gc, line: 48
|
||||
(defmethod reset! collide-sticky-rider-group ((obj collide-sticky-rider-group))
|
||||
[Method] Type: collide-overlap-result Method Name: reset! Defined: text from goal_src/engine/collide/collide-shape-h.gc, line: 94
|
||||
(defmethod reset! collide-overlap-result ((obj collide-overlap-result))
|
||||
[Method] Type: load-state Method Name: reset! Defined: text from goal_src/engine/level/load-boundary.gc, line: 9
|
||||
(defmethod reset! load-state ((obj load-state))
|
||||
|
||||
;; get info about a constant
|
||||
g > (get-info TWO_PI)
|
||||
[Constant] Name: TWO_PI Value: (the-as float #x40c90fda) Defined: text from goal_src/engine/math/trigonometry.gc, line: 34
|
||||
(defconstant TWO_PI (the-as float #x40c90fda))
|
||||
|
||||
;; get info about a built-in form
|
||||
g > (get-info asm-file)
|
||||
[Built-in Form] asm-file
|
||||
```
|
||||
|
||||
### `autocomplete`
|
||||
|
||||
Preview the results of the REPL autocomplete:
|
||||
|
||||
```lisp
|
||||
(autocomplete <sym>)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```lisp
|
||||
g > (autocomplete *)
|
||||
*
|
||||
*16k-dead-pool*
|
||||
*4k-dead-pool*
|
||||
...
|
||||
Autocomplete: 326/1474 symbols matched, took 1.29 ms
|
||||
```
|
||||
|
||||
### `seval`
|
||||
|
||||
Execute GOOS code.
|
||||
|
||||
```lisp
|
||||
(seval form...)
|
||||
```
|
||||
|
||||
Evaluates the forms in the GOOS macro language. The result is not returned in any way, so it's only useful for getting side effects. It's not really used other than to bootstrap some GOAL macros for creating macros.
|
||||
|
||||
### `asm-file`
|
||||
|
||||
Compile a file.
|
||||
|
||||
```lisp
|
||||
(asm-file "file-name" [:color] [:write] [:load] [:no-code])
|
||||
```
|
||||
|
||||
This runs the compiler on a given file. The file path is relative to the `jak-project` folder. These are the options:
|
||||
- `:color`: run register allocation and code generation. Can be omitted if you don't want actually generate code. Usually you want this option.
|
||||
- `:write`: write the object file to the `out/obj` folder. You must also have `:color` on. You must do this to include this file in a DGO.
|
||||
- `:load`: send the object file to the target with the listener. Requires `:color` but not `:write`. There may be issues with `:load`ing very large object files (believed fixed).
|
||||
- `:disassemble`: prints a disassembly of the code by function. Currently data is not disassebmled. This code is not linked so references to symbols will have placeholder values like `0xDEADBEEF`. The IR is printed next to each instruction so you can see what symbol is supposed to be linked. Requires `:color`.
|
||||
- `:no-code`: checks that the result of processing the file generates no code or data. This will be true if your file contains only macros / constant definition. The `goal-lib.gc` file that is loaded by the compiler automatically when it starts must generate no code. You can use `(asm-file "goal_src/goal-lib.gc" :no-code)` to reload this file and double check that it doesn't generate code.
|
||||
|
||||
To reduce typing, there are some useful macros:
|
||||
- `(m "filename")` is "make" and does a `:color` and `:write`.
|
||||
- `(ml "filename")` is "make and load" and does a `:color` and `:write` and `:load`. This effectively replaces the previous version of file in the currently running game with the one you just compiled, and is a super useful tool for quick debugging/iterating.
|
||||
- `(md "filename")` is "make debug" and does a `:color`, `:write`, and `:disassemble`. It is quite useful for working on the compiler and seeing what code is output.
|
||||
- `(build-game)` does `m` on all game files and rebuilds DGOs
|
||||
- `(blg)` (build and load game) does `build-game` then sends commands to load KERNEL and GAME CGOs. The load is done through DGO loading, not `:load`ing individual object files.
|
||||
|
||||
### `asm-data-file`
|
||||
|
||||
Build a data file.
|
||||
|
||||
```lisp
|
||||
(asm-data-file tool-name "file-name")
|
||||
```
|
||||
|
||||
The `tool-name` refers to which data building tool should be used. For example, this should be `game-text` when building the game text data files.
|
||||
|
||||
There's a macro `(build-data)` which rebuilds everything.
|
||||
|
||||
### `gs`
|
||||
|
||||
Enter a GOOS REPL.
|
||||
|
||||
```lisp
|
||||
(gs)
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```scheme
|
||||
g> (gs)
|
||||
goos> (+ 1 2 3)
|
||||
6
|
||||
goos> (exit)
|
||||
()
|
||||
```
|
||||
|
||||
mainly useful for debugging/trying things out in GOOS. The GOOS REPL shares its environment with the GOOS interpreter used by the compiler, so you can inspect/modify things for debugging with this. Likely not used much outside of initial debugging.
|
||||
|
||||
### `set-config!`
|
||||
|
||||
```lisp
|
||||
(set-config! config-name config-value)
|
||||
```
|
||||
|
||||
Used to set compiler configuration. This is mainly for debugging the compiler and enabling print statements. There is a `(db)` macro which sets all the configuration options for the compiler to print as much debugging info as possible. Not used often.
|
||||
|
||||
### `in-package`
|
||||
|
||||
```lisp
|
||||
(in-package stuff...)
|
||||
```
|
||||
|
||||
The compiler ignores this. GOAL files evidently start with this for some reason related to emacs.
|
||||
|
||||
### `build-dgos`
|
||||
|
||||
```lisp
|
||||
(build-dgos "path to dgos description file")
|
||||
```
|
||||
|
||||
Builds all the DGO files described in the DGO description file. See `goal_src/builds/dgos.txt` for an example. This just packs existing things into DGOs - you must have already built all the dependencies.
|
||||
|
||||
In the future, this may become part of `asm-data-file`.
|
||||
|
||||
### `add-macro-to-autocomplete`
|
||||
|
||||
```lisp
|
||||
(add-macro-to-autocomplete macro-name)
|
||||
```
|
||||
|
||||
Makes the given name show up as a macro in the GOAL REPL. Generating macros may be done programmatically using GOOS and this form can be used to make these show up in the autocomplete. This also makes the macro known to `get-info` which will report that the macro was defined at the location where the macro which expanded to `add-macro-to-autocomplete` is located in GOAL code. This is used internally by `defmacro`.
|
||||
@ -1,239 +0,0 @@
|
||||
# OpenGOAL Syntax & Examples
|
||||
|
||||
## The Basics
|
||||
|
||||
### Atoms
|
||||
|
||||
An "atom" in Lisp is a form that can't be broken down into smaller forms. For example `1234` is an atom, but `(1234 5678)` is not. OpenGOAL supports the following atoms:
|
||||
|
||||
### Integers
|
||||
|
||||
All integers are by default `int`, a signed 64-bit integer. You can use:
|
||||
- decimal: Like `123` or `-232`. The allowable range is `INT64_MIN` to `INT64_MAX`.
|
||||
- hex: Like `#x123`. The allowable range is `0` to `UINT64_MAX`. Values over `INT64_MAX` will wrap around.
|
||||
- binary: Like `#b10101010`. The range is the same as hex.
|
||||
- character:
|
||||
- Most can be written like `#\c` for the character `c`.
|
||||
- Space is `#\\s`
|
||||
- New Line is `#\\n`
|
||||
- Tab is `#\\t`
|
||||
|
||||
GOAL has some weird behavior when it comes to integers. It may seem complicated to describe, but it really makes the implementation simpler - the integer types are designed around the available MIPS instructions.
|
||||
|
||||
Integers that are used as local variables (defined with `let`), function arguments, function return values, and intermediate values when combining these are called "register integers", as the values will be stored in CPU registers.
|
||||
|
||||
Integers that are stored in memory as a field of a `structure`/`basic`, an element in an array, or accessed through a `pointer` are "memory integers", as the values will need to be loaded/stored from memory to access them.
|
||||
|
||||
The "register integer" types are `int` and `uint`. They are 64-bit and mostly work exactly like you'd expect. Multiplication, division, and mod, are a little weird and are documented separately.
|
||||
|
||||
The "memory integer" types are `int8`, `int16`, `int32`, `int64`, `uint8`, `uint16`, `uint32`, and `uint64`.
|
||||
|
||||
Conversions between these types are completely automatic - as soon as you access a "memory integer", it will be converted to a "register integer", and trying to store a "register integer" will automatically convert it to the appropriate "memory integer". It (should be) impossible to accidentally get this wrong.
|
||||
|
||||
#### Side Note
|
||||
|
||||
- It's not clear what types `(new 'static 'integer)` or `(new 'stack 'integer)` are, though I would assume both are memory.
|
||||
- If there aren't enough hardware registers, "register integers" can be spilled to stack, but keep their "register integer" types. This process should be impossible to notice, so you don't have to worry about it.
|
||||
|
||||
### String
|
||||
|
||||
A string generates a static string constant. Currently the "const" of this string "constant" isn't enforced. Creating two identical string constants creates two different string objects, which is different from GOAL and should be fixed at some point.
|
||||
|
||||
The string data is in quotes, like in C. The following escapes are supported:
|
||||
- Newline: `\n`
|
||||
- Tab: `\t`
|
||||
- The `\` character: `\\`
|
||||
- The `"` character: `\"`
|
||||
- Any character: `\cXX` where `XX` is the hex number for the character.
|
||||
|
||||
### Float
|
||||
|
||||
Any number constant with a decimal in it. The trailing and leading zeros and negative sign is flexible, so you can do any of these:
|
||||
- `1.`, `1.0`, `01.`, `01.0`
|
||||
- `.1`, `0.1`, `.10`, `0.10`
|
||||
- `-.1`, `-0.1`, `-.10`, `-0.10`
|
||||
|
||||
Like string, it creates a static floating point constant. In later games the float was inlined instead of being a static constant.
|
||||
|
||||
### Symbol
|
||||
|
||||
Use `symbol-name` to get the value of a symbol and `'symbol-name` to get the symbol object.
|
||||
|
||||
### Comments
|
||||
|
||||
Use `;` for line comments and `#|` and `|#` for block comments.
|
||||
|
||||
## Compiling a list
|
||||
|
||||
When the compiler encounters a list like `(a b c)` it attempts to parse in multiple ways in this order:
|
||||
1. A compiler form
|
||||
2. A GOOS macro
|
||||
3. An enum (not yet implemented)
|
||||
4. A function or method call
|
||||
|
||||
## Compiling an integer
|
||||
|
||||
Integers can be specified as
|
||||
- decimal: `1` or `-1234` (range of `INT64_MIN` to `INT64_MAX`)
|
||||
- hex: `#x123`, `#xbeef` (range of `0` to `UINT64_MAX`)
|
||||
- binary: `#b101010` (range of `0` to `UINT64_MAX`)
|
||||
|
||||
All integers are converted to the signed "integer in variable" type called `int`, regardless of how they are specified.
|
||||
Integer "constant"s are not stored in memory but instead are generated by code, so there's no way to modify them.
|
||||
|
||||
## Compiling a string
|
||||
|
||||
A string constant can be specified by just putting it in quotes. Like `"this is a string constant"`.
|
||||
There is an escape code `\` for string:
|
||||
- `\n` newline
|
||||
- `\t` tab character
|
||||
- `\\` the `\` character
|
||||
- `\"` the `"` character
|
||||
- `\cXX` where `XX` is a two character hex number: insert this character.
|
||||
- Any other character following a `\` is an error.
|
||||
|
||||
OpenGOAL stores strings in the same segment of the function which uses the string. I believe GOAL does the same.
|
||||
|
||||
In GOAL, string constants are pooled per object file (or perhaps per segment)- if the same string appears twice, it is only included once. OpenGOAL currently does not pool strings. If any code is found that modifies a string "constant", or if repeated strings take up too much memory, string pooling will be added.
|
||||
|
||||
For now I will assume that string constants are never modified.
|
||||
|
||||
## Compiling a float
|
||||
|
||||
A floating point constant is distinguished from an integer by a decimal point. Leading/trailing zeros are optional. Examples of floats: `1.0, 1., .1, -.1, -0.2`. Floats are stored in memory, so it may be possible to modify a float constant. For now I will assume that float constants are never modified. It is unknown if they are pooled like strings.
|
||||
|
||||
Trivia: Jak 2 realized that it's faster to store floats inline in the code.
|
||||
|
||||
## Compiling a symbol
|
||||
|
||||
A `symbol` appearing in code is compiled by trying each of these in the following order
|
||||
1. Is it `none`? (see section on `none`)
|
||||
2. Try `mlet` symbols
|
||||
3. Try "lexical" variables (defined in `let`)
|
||||
4. Try global constants
|
||||
5. Try global variables (includes named functions and all types)
|
||||
|
||||
## The Special `none` type
|
||||
|
||||
Anything which doesn't return anything has a return type of `none`, indicating the return value can't be used. This is similar to C's `void`.
|
||||
|
||||
## GOAL Structs vs. C Structs
|
||||
|
||||
There is one significant difference between C and GOAL when it comes to structs/classes - GOAL variables can only be references to structs.
|
||||
|
||||
As an example, consider a GOAL type `my-type` and a C type `my_type`. In C/C++, a variable of type `my_type` represents an entire copy of a `my_type` object, and a `my_type*` is like a reference to an existing `my_type` object. In GOAL, an object of `my-type` is a reference to an existing `my-type` object, like a C `my_type*`. There is no equivalent to a C/C++ `my_type`.
|
||||
|
||||
As a result you cannot pass or return a structure by value.
|
||||
|
||||
Another way to explain this is that GOAL structures (including `pair`) always have reference semantics. All other GOAL types have value semantics.
|
||||
|
||||
## Pointers
|
||||
|
||||
GOAL pointers work a lot like C/C++ pointers, but have some slight differences:
|
||||
- A C `int32_t*` is a GOAL `(pointer int32)`
|
||||
- A C `void*` is a GOAL `pointer`
|
||||
- In C, if `x` is a `int32_t*`, `x + 1` is equivalent to `uintptr_t(x) + sizeof(int32_t)`. In GOAL, all pointer math is done in units of bytes.
|
||||
- In C, you can't do pointer math on a `void*`. In GOAL you can, and all math is done in units of bytes.
|
||||
|
||||
In both C and GOAL, there is a connection between arrays and pointers. A GOAL array field will have a pointer-to-element type, and a pointer can be accessed as an array.
|
||||
|
||||
One confusing thing is that a `(pointer int32)` is a C `int32_t*`, but a `(pointer my-structure-type)` is a C `my_structure_type**`, because a GOAL `my-structure-type` is like a C `my_structure_type*`.
|
||||
|
||||
## Inline Arrays
|
||||
|
||||
One limitation of the system above is that an array of `my_structure_type` is actually an array of references to structures (C `object*[]`). It would be more efficient if instead we had an array of structures, laid out together in memory (C `object[]`).
|
||||
|
||||
GOAL has a "inline array" to represent this. A GOAL `(inline-array thing)` is like a C `thing[]`. The inline-array can only be used on structure types, as these are the only reference types.
|
||||
|
||||
## Fields in Structs
|
||||
|
||||
For a field with a reference type (structure/basic)
|
||||
- `(data thing)` is like C `Thing* data;`
|
||||
- `(data thing :inline #t)` is like C `Thing data;`
|
||||
- `(data thing 12)` is like C `Thing* data[12];`. The field has `(pointer thing)` type.
|
||||
- `(data thing 12 :inline #t)` is like `Thing data[12];`. The field has `(inline-array thing)` type
|
||||
|
||||
For a field with a value type (integer, etc)
|
||||
- `(data int32)` is like C `int32_t data;`
|
||||
- `(data int32 12)` is like `int32_t data[12];`. The field has `(array int32)` type.
|
||||
|
||||
Using the `:inline #t` option on a value type is not allowed.
|
||||
|
||||
## Dynamic Structs
|
||||
|
||||
GOAL structure can be dynamically sized, which means their size isn't determined at compile time. Instead the user should implement `asize-of` to return the actual size.
|
||||
|
||||
This works by having the structure end in an array of unknown size at compile time. In a dynamic structure definition, the last field of the struct should be an array with an unspecified size. To create this, add a `:dynamic #t` option to the field and do not specify an array size. This can be an array of value types, an array of reference types, or an inline-array of reference types.
|
||||
|
||||
### Unknown
|
||||
|
||||
Is the `size` of a dynamic struct:
|
||||
- size assuming the dynamic array has 0 elements (I think it's this)
|
||||
- size assuming the dynamic array doesn't
|
||||
|
||||
These can differ by padding for alignment.
|
||||
|
||||
|
||||
## How To Create GOAL Objects - `new`
|
||||
|
||||
GOAL has several different ways to create objects, all using the `new` form.
|
||||
|
||||
### Heap Allocated Objects
|
||||
|
||||
A new object can be allocated on a heap with `(new 'global 'obj-type [new-method-arguments])`.
|
||||
This simply calls the `new` method of the given type. You can also replace `'global` with `'debug` to allocate on the debug heap.
|
||||
Currently these are the only two heaps supported, in the future you will be able to call the new method with other arguments
|
||||
to allow you to do an "in place new" or allocate on a different heap.
|
||||
|
||||
This will only work on structures and basics. If you want a heap allocated float/integer/pointer, create an array of size 1.
|
||||
This will work on dynamically sized items.
|
||||
|
||||
### Heap Allocated Arrays
|
||||
|
||||
You can construct a heap array with `(new 'global 'inline-array 'obj-type count)` or `(new 'global 'array 'obj-type count)`.
|
||||
These objects are not initialized. Note that the `array` version creates a `(pointer obj-type)` plain array,
|
||||
__not__ a GOAL `array` type fancy array. In the future this may change because it is confusing.
|
||||
|
||||
Because these objects are uninitialized, you cannot provide constructor arguments.
|
||||
You cannot use this on dynamically sized member types. However, the array size can be determined at runtime.
|
||||
|
||||
### Static Objects
|
||||
|
||||
You can create a static object with `(new 'static 'obj-type [field-def]...)`. It can be a structure, basic, bitfield, array, boxed array, or inline array.
|
||||
Each field def looks like `:field-name field-value`. The `field-value` is evaluated at compile time. Fields
|
||||
can be integers, floats, symbols, pairs, strings, or other statics. These field values may come from macros or GOAL constants.
|
||||
|
||||
For bitfields, there is an exception, and fields can be set to expression that are not known at compile time. The compiler will generate the appropriate code to combine the values known at compile time and run time. This exception does not apply to a bitfield inside of another `(new 'static ...)`.
|
||||
|
||||
Fields which aren't explicitly initialized are zeroed, except for the type field of basics, which is properly initialized to the correct type.
|
||||
|
||||
This does not work on dynamically sized structures.
|
||||
|
||||
### Stack Allocated Arrays
|
||||
|
||||
Currently only arrays of integers, floats, or pointers can be stack allocated.
|
||||
For example, use `(new 'stack ''array 'int32 1)` to get a `(pointer int32)`. Unlike heap allocated arrays, these stack arrays
|
||||
must have a size that can be determined at compile time. The objects are uninitialized.
|
||||
|
||||
### Stack Allocated Structures
|
||||
|
||||
Works like heap allocated, the objects are initialized with the constructor. The constructor must support "stack mode". Using `object-new` supports stack mode so usually you don't have to worry about this. The structure's memory will be memset to 0 with `object-new` automatically.
|
||||
|
||||
### Defining a `new` Method
|
||||
|
||||
TODO
|
||||
|
||||
## Array Spacing
|
||||
|
||||
In general, all GOAL objects are 16-byte aligned and the boxing system requires this. All heap memory allocations are 16-byte aligned too, so this is usually not an issue.
|
||||
|
||||
## Truth
|
||||
|
||||
Everything is true except for `#f`. This means `0` is true, and `'()` is true.
|
||||
The value of `#f` can be used like `nullptr`, at least for any `basic` object. It's unclear if `#f` can/should be used as a null for other types, including `structure`s or numbers or pointers.
|
||||
|
||||
Technical note: the hex number `0x147d24` is considered false in Jak 1 NTSC due to where the symbol table happened to be allocated. However, checking numbers for true/false shouldn't be done, you should use `(zero? x)` instead.
|
||||
|
||||
## Empty Pair
|
||||
|
||||
TODO
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user