add tex/lyx original versions of archival PDFs (#227)
* restore underscore, update OpenGL dev links * add tex/lyx versions of archival documentation * restore underscore, update OpenGL dev links (#223) * organize development docs TOC / add git file format directives (#222) * organize developer TOC * add git file format metadata/directives * add "skeletor" to core dev doc; refactor text * add Vectrexia development guide to core dev docs * sort compilation guide TOC into cores and frontend
80
archive/libretro-gl.tex
Normal file
@ -0,0 +1,80 @@
|
||||
\documentclass[11pt]{article}
|
||||
|
||||
\author{Hans-Kristian Arntzen}
|
||||
\title{Implementing a Hardware Accelerated Libretro Core}
|
||||
|
||||
\begin{document}
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
Libretro API~\footnote{http://libretro.org} recently received support for cores to use OpenGL (GL2+ or GLES2) directly instead of software rendering to a buffer which is subsequently used by the frontend. This article explains how a core can take advantage of this, and which considerations must be taken into account. This article assumes familiarity with the libretro API.
|
||||
|
||||
Cores which use hardware rendering can still use known frontend features, such as multi-pass shaders.
|
||||
This is accomplished by letting cores render to frame buffer objects (FBOs) instead of the back buffer~\footnote{GL drivers must support render-to-texture extensions for this to work.}.
|
||||
|
||||
This addition to the libretro API is designed to be used with hardware-accelerated emulator cores, as well as serving as a framework for graphical demos and experiments.
|
||||
\end{abstract}
|
||||
|
||||
\section*{Application model}
|
||||
Using OpenGL in a libretro context is somewhat different than when you use libraries like SDL, GLFW or SFML. In libretro, the frontend owns the OpenGL context. For an application using conventional libraries like SDL, the application will do this:
|
||||
|
||||
\begin{itemize}
|
||||
\item Initialize, create a window of specific size
|
||||
\item Initialize OpenGL resources
|
||||
\item Per frame, handle window events (resize), handle input, render as desired, swap buffers
|
||||
\item Tear down context and window
|
||||
\end{itemize}
|
||||
|
||||
Using libretro API, platform specifics like managing windows, rendering surfaces and input are all handled by the frontend. The core will only deal with rendering to a surface. The core renders to an FBO of fixed size, determined by the core. The frontend takes this rendered data and stretches to screen as desired by the user. It can apply shaders, change aspect ratio, etc. This model is equivalent to software rendering where \texttt{retro\_video\_refresh\_t} callback is called.
|
||||
|
||||
\section*{Using OpenGL in libretro}
|
||||
|
||||
\begin{itemize}
|
||||
\item Use \texttt{RETRO\_ENVIRONMENT\_SET\_PIXEL\_FORMAT} and request a 32-bit format. This is the format that the resulting framebuffer will have~\footnote{In reality, RetroArch converts all 16-bit data (\texttt{RETRO\_PIXEL\_FORMAT\_RGB565}) to 32-bit (\texttt{XRGB8888}) when running desktop GL for performance reasons. In GLES mode, this is not done, however. Do not rely on this behavior, and be explicit about it.}.
|
||||
|
||||
\item Use \texttt{RETRO\_ENVIRONMENT\_SET\_HW\_RENDER} environment callback in \\\texttt{retro\_load\_game()}, notifying frontend that core is using hardware rendering. An OpenGL 2+ or GLES2 context can be specified here. If this is not supported the callback will return false, and you can fallback to software rendering or refuse to start.
|
||||
|
||||
\item In \texttt{retro\_get\_system\_av\_info()}, as normal, \texttt{max\_width} and \\\texttt{max\_height} fields specify the maximum resolution the core will render to.
|
||||
|
||||
\item When the frontend has created a context or reset the context, \\\texttt{retro\_hw\_context\_reset\_t} is called. Here, OpenGL resources can be initialized.
|
||||
The frontend can reset the context at will (e.g. when changing from fullscreen to windowed mode and vice versa). The core should take this into account. It will be notified when reinitialization needs to happen.
|
||||
|
||||
\item A callback to grab OpenGL symbols is exposed via \\\texttt{retro\_hw\_get\_proc\_address\_t}. Use this to retrieve symbols and extensions.
|
||||
|
||||
\item In \texttt{retro\_run()}, use \texttt{retro\_hw\_get\_current\_framebuffer\_t} callback
|
||||
to get which FBO to render to~\footnote{e.g. \texttt{glBindFramebuffer(GL\_FRAMEBUFFER, get\_current\_framebuffer())}}. This is your "backbuffer". Do not attempt to render to the real back buffer. You must call this every frame as it can change every frame. The dimensions of this FBO are at least as big as declared in \texttt{max\_width} and \texttt{max\_height}. If desired, the FBO also has a depth buffer attached~\footnote{see \texttt{RETRO\_ENVIRONMENT\_SET\_HW\_RENDER}}.
|
||||
|
||||
\item When done rendering, call \texttt{retro\_video\_refresh\_t} with the macro \texttt{RETRO\_HW\_FRAME\_BUFFER\_VALID} as argument for buffer. Width and height should be specified as well, but pitch argument is irrelevant and will be ignored. If the frame is duped~\footnote{RETRO\_ENVIRONMENT\_CAN\_DUPE}, the buffer argument takes \texttt{NULL} as normal.
|
||||
\end{itemize}
|
||||
|
||||
\section*{Important considerations in the OpenGL code}
|
||||
The frontend and libretro core share OpenGL context state. Some considerations have to be taken into account for this cooperation to work nicely.
|
||||
|
||||
\begin{itemize}
|
||||
\item Don't leave buffers and global objects bound when calling \\\texttt{retro\_video\_refresh\_t}.
|
||||
Make sure to unbind everything, i.e. VAOs, VBOs, shader programs, textures, etc. Failing to do this could potentially hit strange bugs. The frontend will also follow this rule to avoid clashes. Being tidy here is considered good practice anyway.
|
||||
|
||||
\item The GL viewport will be modified by frontend as well as libretro core. Set this every frame.
|
||||
|
||||
\item \texttt{glEnable()} state like depth testing, etc, is likely to be disabled in frontend as it's just rendering a quad to screen. Enable this per-frame if you use depth testing.
|
||||
There is no need to disable this before calling \\\texttt{retro\_video\_refresh\_t}.
|
||||
|
||||
\item Avoid VAOs. They tend to break on less-than-stellar drivers~\footnote{At least AMD drivers on Windows are known to break here.}.
|
||||
|
||||
\item Try to write code which is GLES2 as well as GL2+ (w/ extensions) compliant. This ensures maximum target surface for the libretro core.
|
||||
|
||||
\item Libretro treats top-left as origin. OpenGL treats bottom-left as origin. To be compatible with the libretro model, top-left semantics are preserved. Rendering normally will cause the image to be flipped vertically. To avoid this, simply scale the final projection matrix by $[1, -1, 1, 1]$.
|
||||
\end{itemize}
|
||||
|
||||
\section*{Test implementations}
|
||||
A very basic test implementation of libretro GL interface is available in RetroArch repository on GitHub~\footnote{https://github.com/Themaister/RetroArch/tree/master/libretro-test-gl}.
|
||||
It displays two spinning quads. It runs both as a GLES2 and GL2 core depending on \texttt{GLES} environment variable.
|
||||
|
||||
A slightly more involved test core is found on Bitbucket~\footnote{https://bitbucket.org/Themaister/libretro-gl}. It uses instanced rendering of a textured cube, with FPS-style fly-by camera. It uses libretro's mouse API as well. It is valid GLES and GL2 at the same time.
|
||||
|
||||
\section*{Building a libretro core}
|
||||
Libretro is an interface, and not a utility library. Libretro cores are built as standalone dynamic or static libraries, and as they use GL symbols here, they must link against GL symbols themselves.
|
||||
|
||||
An example of how this can be done is shown in the test implementation~\footnote{https://github.com/Themaister/RetroArch/blob/master/libretro-test-gl/Makefile}.
|
||||
|
||||
\end{document}
|
2192
archive/libretro-shader.lyx
Normal file
1169
archive/libretro.lyx
Normal file
BIN
archive/mario-water1.jpg
Normal file
After Width: | Height: | Size: 56 KiB |
BIN
archive/mario-water2.jpg
Normal file
After Width: | Height: | Size: 67 KiB |
115
archive/overlay.tex
Normal file
@ -0,0 +1,115 @@
|
||||
\documentclass[a4paper, 11pt]{article}
|
||||
|
||||
\title{RetroArch - Overlay image configuration}
|
||||
\author{
|
||||
Hans-Kristian Arntzen
|
||||
\and
|
||||
Daniel De Matteis}
|
||||
|
||||
\begin{document}
|
||||
\maketitle
|
||||
|
||||
\section*{Purpose}
|
||||
|
||||
RetroArch supports overlay images for use with hardware accelerated drivers.
|
||||
The purpose of this is to allow some kind of input interface that is mouse/touch oriented.
|
||||
|
||||
The overlay image is displayed with transparency over the regular game image, and the user is able to trigger input by pressing on certain parts of the overlay.
|
||||
|
||||
Since the overlay is an image, the user should be able to fully configure the look and functionality of this overlay. This allows skinners and themers to go wild.
|
||||
|
||||
\section*{Configuration}
|
||||
|
||||
The overlay is described in a config file (*.cfg). The config file uses the same config file syntax as RetroArch itself.
|
||||
|
||||
The overlay system supports use of multiple overlays that can be switched out on the fly. Input is not only restricted to gamepad input, but can also work with any input that is bindable in RetroArch, e.g. save states, rewind, load state, etc.
|
||||
|
||||
The config file describes:
|
||||
\begin{itemize}
|
||||
\item Which overlay images to use (.png, .tga, etc).
|
||||
\item Which coordinates on each overlay correspond to which input event.
|
||||
\item The hitbox for each input event, i.e. "size" of the button.
|
||||
\item Where on the screen the overlay should be displayed.
|
||||
\end{itemize}
|
||||
|
||||
|
||||
\subsection*{Overlay images}
|
||||
|
||||
First we configure how many overlays we use, and where they can be found.
|
||||
|
||||
\begin{verbatim}
|
||||
overlays = 2
|
||||
overlay0_overlay = overlay_img0.png
|
||||
overlay1_overlay = overlay_img1.png
|
||||
\end{verbatim}
|
||||
|
||||
The paths are relative to where the overlay config is loaded from. If the path is absolute, absolute paths will be used. On Unix-like systems '~/' is recognized as '\$HOME'.
|
||||
|
||||
\subsection*{Screen placement}
|
||||
|
||||
By default, the overlay image will be stretched out to fill the whole game image.
|
||||
However, for some overlays, this is not practical.
|
||||
|
||||
It is possible to set the placement using for example:
|
||||
|
||||
\begin{verbatim}
|
||||
overlay0_rect = "0.2,0.3,0.5,0.4"
|
||||
\end{verbatim}
|
||||
|
||||
We assume that the game screen has normalized coordinates in X and Y that span from `[0, 0]`
|
||||
in the top-left corner to $[1, 1]$ in the lower-right corner.
|
||||
|
||||
This will render the overlay to $x = 0.2, y = 0.3$ with size $width = 0.5, height = 0.4$.
|
||||
The default (stretch to full screen) could be described as such:
|
||||
|
||||
\begin{verbatim}
|
||||
overlay0_rect = "0.0,0.0,1.0,1.0"
|
||||
\end{verbatim}
|
||||
|
||||
\subsection*{Full-screen overlays}
|
||||
|
||||
By default, overlays will be stretched out to fill game viewport. However, in some cases
|
||||
the aspect ratio of the game causes there to remain large black borders around the game image.
|
||||
It is possible to stretch the overlay to full screen (instead of viewport) by specifying this option:
|
||||
|
||||
\begin{verbatim}
|
||||
overlay0_full_screen = true
|
||||
\end{verbatim}
|
||||
|
||||
\subsection*{Coordinate descriptors}
|
||||
|
||||
We must also describe where on the overlay image buttons can be found for each overlay.
|
||||
E.g.:
|
||||
|
||||
\begin{verbatim}
|
||||
overlay0_descs = 3 # Three buttons for this overlay in total
|
||||
overlay0_desc0 = "a,32,64,radial,10,20"
|
||||
overlay0_desc1 = "start,100,50,rect,80,10"
|
||||
overlay0_desc2 = "overlay_next,200,180,radial,40,40"
|
||||
\end{verbatim}
|
||||
|
||||
The format is:
|
||||
|
||||
\begin{verbatim}
|
||||
"button,position_x,position_y,hitbox_type,range_x,range_y"
|
||||
\end{verbatim}
|
||||
|
||||
'button' corresponds to the input event being generated. The names are the same as in the general input config, e.g. input\_player1\_start would translate to start here. overlay\_next is a special bind designed to swap to the next overlay, or wrap around to the first one.
|
||||
|
||||
'position\_x' and 'position\_y' are the x and y coordinates in pixels of the source image for the center of the button.
|
||||
|
||||
'hitbox\_type' describes which type of shape the button has. 'radial' (circle, ellipsis) and 'rect' (rectangular) shapes are supported.
|
||||
|
||||
'range\_x' and 'range\_y' describe the size of the button. The semantics differ slightly for radial and rect hitbox types. For 'radial' shape, 'range\_x' and 'range\_y' describe the radius in x and y directions in pixels of the source image. For 'rect' shape, the 'range\_x' and 'range\_y' values represent the distance from center to the edge of the rect in terms of pixels of the source image. E.g. a 'range\_x' of 20 would mean the width of the rectangular is 40 pixels in total.
|
||||
|
||||
\subsection*{Triggering multiple buttons with one desc}
|
||||
|
||||
It's possible to trigger multiple buttons (e.g. diagonals) with one overlay desc.
|
||||
|
||||
\begin{verbatim}
|
||||
overlay0_desc0 = "left|up,32,64,radial,10,20"
|
||||
\end{verbatim}
|
||||
|
||||
will trigger both left and up at same time.
|
||||
|
||||
\end{document}
|
BIN
archive/ratecontrol-data.pdf
Normal file
211
archive/ratecontrol.tex
Normal file
@ -0,0 +1,211 @@
|
||||
\documentclass[11pt, a4paper]{article}
|
||||
|
||||
\title{Dynamic Rate Control for Retro Game Emulators}
|
||||
\author{Hans-Kristian Arntzen}
|
||||
|
||||
\usepackage{amsmath}
|
||||
\usepackage{float}
|
||||
\usepackage{graphicx}
|
||||
|
||||
\begin{document}
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
This article describes a method for game emulator frontends
|
||||
to synchronize both audio and video output at the same time, even
|
||||
when the emulating system has a different refresh rate and audio sampling rate
|
||||
than the gaming system that is being emulated.
|
||||
|
||||
The method works by dynamically adjusting audio resampling ratios in such ways that
|
||||
ideally, the audio buffer is never underrun nor overrun, thus avoiding blocking on audio.
|
||||
This in turn allows vertical synchronization for video.
|
||||
|
||||
The audio pitch is adjusted when adjusting audio resampling ratios,
|
||||
but in practice so little, that it is inaudible to the human ear.
|
||||
\end{abstract}
|
||||
|
||||
\section{Background}
|
||||
|
||||
Retro game consoles are highly synchronous. Their audio output rates are linked directly
|
||||
to video refresh rates. Every video frame, the audio chip generates on average a fixed amount of audio samples. Before continuing to emulate the next frame, the generated audio samples must be pushed
|
||||
to an audio buffer of fixed size.
|
||||
|
||||
If there is not enough space in the audio buffer, the emulator must wait (block) for the buffer to become ready for writing. This is a non-ideal situation as while the emulator is blocking on audio, a vertical refresh might be missed entirely, thus creating stuttering video.
|
||||
|
||||
\subsection{The ideal synchronization}
|
||||
For an emulator of a retro game system, a key factor in smooth video is vertical refresh synchronization (VSync), where each frame of the game maps to a single frame on the monitor.
|
||||
Audio must also be pushed to the speakers without any audio dropouts.
|
||||
This double synchronization requirement poses a problem as any form of synchronization to one modality
|
||||
will negatively affect the other.
|
||||
|
||||
This is a real problem as an emulator has no way of guaranteeing perfectly equal
|
||||
video refresh rates and audio sampling rates as the original system.
|
||||
|
||||
On conventional computer hardware, there is no perfect way of knowing
|
||||
the real monitor refresh rates and audio sampling rates either due to
|
||||
tolerances on oscillators.
|
||||
|
||||
\subsection{Scope of method}
|
||||
As this method aims to implement a method for synchronization when VSync is used,
|
||||
this method is only useful when game frame rate is close to monitor frame rate.
|
||||
If this is not the case, other methods should be employed.
|
||||
|
||||
\section{Method}
|
||||
This method assumes that audio from the emulator is output at regular intervals, e.g.
|
||||
every video frame. The method also assumes that audio is resampled from
|
||||
the game system sampling rate to the sound cards sampling rate.
|
||||
The resampling ratio will be dynamically adjusted every time audio is resampled and subsequently pushed to the audio buffer.
|
||||
|
||||
\subsection{Definitions}
|
||||
\begin{itemize}
|
||||
\item[$f_v$] Emulated game system frame rate (FPS)
|
||||
\item[$f_a$] Emulated game system sampling rate (Hz)
|
||||
\item[$r$] Emulated game system samples per frame $f_a / f_v$
|
||||
\item[$m_a$] Emulator system sampling rate
|
||||
\item[$m_v$] Emulator system monitor refresh rate
|
||||
\item[$m_a^{'}$] Estimated emulator system sampling rate
|
||||
\item[$m_v^{'}$] Estimated emulator system monitor refresh rate
|
||||
\item[$R$] Emulator system samples per frame $m_a / m_v$
|
||||
\item[$R^{'}$] Estimated emulator system samples per frame $m_a^{'} / m_v^{'}$
|
||||
\item[$A_b$] Current amount of samples in the audio buffer
|
||||
\item[$A_B$] Capacity (in samples) of the audio buffer
|
||||
\item[$d$] Allowed deviation in audio pitch
|
||||
\end{itemize}
|
||||
|
||||
\subsection{Resampling audio}
|
||||
Every time the game system outputs audio, it is resampled with some ratio.
|
||||
It is here assumed that the game system outputs a video frame worth of audio at a time.
|
||||
While the formulae are invariant to how often audio is written to audio buffers, this assumption
|
||||
is made for simplicity.
|
||||
|
||||
The correct resampling ratio is estimated to be $R^{'} / r$,
|
||||
thus pushing $R^{'}$ samples of audio per frame on average.
|
||||
In the duration of a frame, on average, $R$ audio frames will have been played to the speakers
|
||||
and thus removed from the buffer.
|
||||
|
||||
\begin{equation}
|
||||
\Delta A_b = R^{'} - R
|
||||
\end{equation}
|
||||
|
||||
We see that unless $\Delta A_b = 0$, it is inevitable that the audio will either underrun ($A_b \leq 0$),
|
||||
or block due to buffer being full ($A_b \geq A_B$).
|
||||
Both these situations are not acceptable as underruns would cause audible audio dropouts,
|
||||
and blocking would block the emulator from emulating more frames, and thus greatly increasing the chance of missing a VBlank, which is not acceptable as well. Blocking however, is far more preferable than underrunning.
|
||||
|
||||
As with any estimator, it is impossible to guarantee that $R^{'}$ can be perfectly estimated.
|
||||
Therefore, it is impossible to guarantee that underrun nor blocking occurs.
|
||||
|
||||
\subsection{Dynamic rate control}
|
||||
The proposed method will dynamically adjust the resampling ratio $R^{'} / r$.
|
||||
Changing this ratio will adjust audio pitch as well. To ensure that these adjustments are not audible to the human ear, the range of adjustment will be limited by $d$.
|
||||
|
||||
Using $d$, the maximum pushed samples will be $R^{'} \left(1 + d\right)$,
|
||||
and similarly, minimum will be $R^{'} \left(1 - d\right)$.
|
||||
$d$ must be chosen so that $R$ falls between minimum or maximum. This depends on the confidence of the estimate $R^{'}$.
|
||||
|
||||
The revised update formula will look like this:
|
||||
|
||||
\begin{equation} \label{eq:update}
|
||||
\Delta A_b = \left[ 1 + \left(\frac{A_B - 2A_b}{A_B}\right) d \right] R^{'} - R
|
||||
\end{equation}
|
||||
|
||||
The formula will decrease resampling ratio if audio buffer is over half full, and similarly increase
|
||||
resampling ratio if buffer is below half full.
|
||||
|
||||
\subsection{Stability}
|
||||
|
||||
To ensure that the method is stable, i.e. that $A_b$ will converge to a certain value,
|
||||
we assume a continuous model for pushing audio.
|
||||
|
||||
\begin{equation}
|
||||
\frac{\delta A_b}{\delta f} = \left[ 1 + \left(\frac{A_B - 2A_b}{A_B}\right) d \right] R^{'} - R
|
||||
\end{equation}
|
||||
\begin{equation} \label{eq:ab-diff}
|
||||
\frac{\delta A_b}{\delta f} + \frac{2dR^{'}}{A_B}A_b = R^{'} \left(1 + d\right) - R
|
||||
\end{equation}
|
||||
|
||||
The differential equation in \eqref{eq:ab-diff} can be solved as
|
||||
|
||||
\begin{equation}
|
||||
A_b = A_B \frac{R^{'}\left(1 + d\right) - R}{2dR^{'}} + C_0\exp \left(-\frac{2dR^{'}}{A_B} f\right)
|
||||
\end{equation}
|
||||
|
||||
Given time ($f \rightarrow \infty$), this expression converges to
|
||||
|
||||
\begin{equation}
|
||||
A_{b,c} = A_B \frac{R^{'}\left(1 + d\right) - R}{2dR^{'}}
|
||||
\end{equation}
|
||||
|
||||
If $R^{'}$ is the ideal estimate, $R^{'} = R$, the expression converges to
|
||||
|
||||
\begin{equation}
|
||||
A_{b,c} = A_B / 2
|
||||
\end{equation}
|
||||
which is the best case, as having a half full buffer means most possible wiggle room for jitter.
|
||||
|
||||
\subsection{Updating ratio estimate $R^{'}$}
|
||||
After time, it is assumed that $A_b$ will converge to $A_{b,c}$.
|
||||
Due to jitter, and various non-ideal behavior, only an estimate of $A_{b,c}$, $\hat{A_{b,c}} = \bar{A_b}$ can be obtained.
|
||||
|
||||
\begin{equation}
|
||||
\frac{\hat{R^{'}}}{R} = \frac{A_B}{\left(1 + d\right) A_B - 2d\hat{A_{b,c}}}
|
||||
\end{equation}
|
||||
|
||||
The ratio estimate $R^{'}$ can thus be re-estimated accordingly. This method of re-estimating the ratio might however not be the best. Directly estimating $m_a^{'}$ and $m_v^{'}$ should yield more confident results, but this is outside the scope of this article.
|
||||
|
||||
\section{Results}
|
||||
|
||||
A synthetic test was carried out to test how the method would react to a common scenario for this method. An emulation of Super Nintendo Entertainment System (SNES) was matched to an emulating system.
|
||||
To test effects of jitter, the frame time was assumed to follow a normal distribution.
|
||||
Thus, $R$ in \eqref{eq:update} followed a normal distribution.
|
||||
|
||||
\begin{table}[H]
|
||||
\centering
|
||||
\caption{Timings}
|
||||
\begin{tabular}{|c|c|c|}
|
||||
\hline
|
||||
System & FPS (Hz) & Sample rate (Hz)\\\hline
|
||||
SNES & 60.0988 & 32040.5\\\hline
|
||||
Emulating system & 59.88 & 48000.15\\\hline
|
||||
Estimated system & 59.95 & 48000.0\\\hline
|
||||
\end{tabular}
|
||||
\end{table}
|
||||
|
||||
\begin{table}[H]
|
||||
\centering
|
||||
\caption{Test parameters}
|
||||
\begin{tabular}{|c|c|}
|
||||
\hline
|
||||
$d$ & $0.005$\\\hline
|
||||
Frame time deviation & $2 \%$\\\hline
|
||||
\end{tabular}
|
||||
\end{table}
|
||||
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=12cm]{ratecontrol-data.pdf}
|
||||
\caption{Results with 2\% standard deviation on frame time}
|
||||
\end{figure}
|
||||
|
||||
The results are shown for audio buffer size over time and the pitch modulation for every simulated frame. Audio pitch modulation deviation was estimated to $0.062\%$, significantly lower than the $d$ value, which allowed for a maximum deviation of $0.5\%$.
|
||||
|
||||
The audio buffer is never filled nor underrun, which would allow every single VBlank to be met, while maintaining audio sync.
|
||||
|
||||
\section{Discussion}
|
||||
|
||||
\subsection{Effect of $d$}
|
||||
Ideally, $d$ should be as low as possible to avoid large deviations in pitch, but at the same time, a too low value for $d$ will not be able to compensate for the difference between $R^{'}$ and $R$.
|
||||
From testing in the emulator frontend RetroArch\footnote{https://github.com/Themaister/RetroArch}, a factor of $d \geq 0.002, d \leq 0.005$ has been found to give satisfactory results.
|
||||
|
||||
This method does not propose a method of determining the best $d$ factor.
|
||||
|
||||
\subsection{Audibility}
|
||||
As audio pitch is altered, there is a question of audibility. Given a small enough $d$, it's
|
||||
clear that the effect would be completely inaudible as all oscillators in a DAC have tolerances and jitter to some degree. It is also reasonable to believe that some persons are able to notice a smaller $d$ than others.
|
||||
|
||||
Some testing must be carried out to find a $d$ that is inaudible in subjective tests.
|
||||
|
||||
\section{Conclusion}
|
||||
This method shows that it is possible to obtain synchronization to both VBlank and audio without having a perfect estimate of the emulating systems frequencies. As long as the estimates are reasonable close to the real values, the dynamic rate control method proposed here is able to smooth out the differences.
|
||||
|
||||
\end{document}
|
2002
archive/retroarch-cores-manual.lyx
Normal file
2792
archive/retroarch-enduserguide.lyx
Normal file
BIN
archive/retroarch-enduserguide.pdf
Normal file
1832
archive/retroarch-manual.lyx
Normal file
BIN
archive/shader-rarch-2.jpg
Normal file
After Width: | Height: | Size: 83 KiB |
BIN
archive/supermetroid.jpg
Normal file
After Width: | Height: | Size: 89 KiB |
BIN
docs/image/development/shaders/content-aware-shader-1.jpg
Normal file
After Width: | Height: | Size: 56 KiB |
Before Width: | Height: | Size: 270 KiB |
BIN
docs/image/development/shaders/content-aware-shader-2.jpg
Normal file
After Width: | Height: | Size: 67 KiB |
Before Width: | Height: | Size: 402 KiB |
@ -146,7 +146,7 @@ To develop these kinds of shaders, I’d recommend using RetroArch w/Cg support,
|
||||
Here are some screenshots of the mario effect (in Super Mario World SNES) we developed. Obviously this is a very simple example showing what can be done. The imagination is the limit here.
|
||||
|
||||
### Prior to Mario jumping in water
|
||||
![Super Mario World prior to Mario jumping in water.](../image/development/shaders/content-aware-shader-1.png)
|
||||
![Super Mario World prior to Mario jumping in water.](../image/development/shaders/content-aware-shader-1.jpg)
|
||||
|
||||
### After Mario jumps in water
|
||||
![Super Mario World after Mario jumps in water](../image/development/shaders/content-aware-shader-2.png)
|
||||
![Super Mario World after Mario jumps in water](../image/development/shaders/content-aware-shader-2.jpg)
|
||||
|
@ -4,7 +4,32 @@ Dynamic Rate Control allows emulator frontends to synchronize both audio and vid
|
||||
|
||||
The method works by dynamically adjusting audio resampling ratios in such ways that ideally, the audio buffer is never underrun nor overrun, thus avoiding blocking on audio. This in turn allows vertical synchronization for video. The audio pitch is adjusted when adjusting audio resampling ratios, but in practice so little, that it is inaudible to the human ear.
|
||||
|
||||
### Read this documenation in PDF form
|
||||
!!! Tip "Read this documenation in PDF form"
|
||||
Because the formulas in this documenation have not yet been converted to markdown, [please consult the original PDF version](https://github.com/libretro/docs/blob/master/archive/ratecontrol.pdf). _If you can assist in the conversion, please post an issue or PR in [the libretro documentation repository](https://github.com/libretro/docs)._
|
||||
|
||||
Because this documenation has not yet been converted to the new format, [please consult the original PDF version](https://github.com/libretro/docs/blob/master/archive/ratecontrol.pdf).
|
||||
Retro games are highly synchronous. Their audio output rates are linked directly to video refresh rates. Every video frame, the audio chip generates on average a fixed amount of audio samples. Before continuing to emulate the next frame, the generated audio samples must be pushed to an audio buffer of fixed size.
|
||||
|
||||
If there is not enough space in the audio buffer, the emulator must wait (block) for the buffer to become ready for writing. This is a non-ideal situation as while the emulator is blocking on audio, a vertical refresh might be missed entirely, thus creating stuttering video.
|
||||
|
||||
## Ideal synchronization
|
||||
|
||||
For an emulator of a retro game system, a key factor in smooth video is vertical refresh synchronization (**VSync**), where each frame of the game maps to a single frame on the monitor. Audio must also be pushed to the speakers without any audio dropouts. This double synchronization requirement poses a problem as any form of synchronization to one modality will negatively affect the other.
|
||||
|
||||
This is a real problem as an emulator has no way of guaranteeing perfectly equal video refresh rates and audio sampling rates as the original system.
|
||||
|
||||
On conventional computer hardware, there is no perfect way of knowing the real monitor refresh rates and audio sampling rates either due to tolerances on oscillators.
|
||||
|
||||
|
||||
## Scope of method
|
||||
|
||||
As this method aims to implement a method for synchronization when VSync is used, this method is only useful when game frame rate is close to monitor frame rate. If this is not the case, other methods should be employed.
|
||||
|
||||
|
||||
## Method
|
||||
|
||||
This method assumes that audio from the emulator is output at regular intervals, e.g. every video frame. The method also assumes that audio is resampled from the game system sampling rate to the sound cards sampling rate. The resampling ratio will be dynamically adjusted every time audio is resampled and subsequently pushed to the audio buffer.
|
||||
|
||||
|
||||
## Link to full version
|
||||
|
||||
Because the formulas in this documenation have not yet been converted to markdown, [please consult the original PDF version](https://github.com/libretro/docs/blob/master/archive/ratecontrol.pdf). _If you can assist in the conversion, please post an issue or PR in [the libretro documentation repository](https://github.com/libretro/docs)._
|