mirror of
https://github.com/BepInEx/mono.git
synced 2026-11-17 11:21:39 -10:00
274 lines
9.2 KiB
Plaintext
274 lines
9.2 KiB
Plaintext
* Testing
|
|
|
|
Daily <a href="http://www.go-mono.com/tests/index.php">test</a> results.
|
|
|
|
Testing is an important part of the Mono project: every one of its
|
|
three major components has a test suite tailored for its needs. This
|
|
is very helpful, because in the course of developing the software it
|
|
is very common to introduce bugs in existing code. A test suite
|
|
helps us fix the bugs as soon as they are introduced.
|
|
|
|
There are various kinds of tests in Mono:
|
|
<ul>
|
|
<li><a href="#unit"><b>Class Library Unit
|
|
Tests:</b></a> These are used to test the class
|
|
libraries.
|
|
|
|
<li><a href="#compiler"><b>Compiler tests</b></a>: Both
|
|
tests that should pass and tests that should fail are included.
|
|
|
|
<li><a href="#runtime"><b>Runtime tests</b></a>: Tests for
|
|
the virtual machine.
|
|
|
|
<li><a href="#aspnet"><b>ASP.NET tests</b></a>: ASP.NET tests.
|
|
|
|
<li><a href="#ws"><b>Web Services tests</b></a>: Web Services
|
|
client/server tests.
|
|
</ul>
|
|
|
|
<a name="unit"></a>
|
|
* Class Library Tests
|
|
|
|
All classes in Mono libraries should have comprehensive unit test
|
|
suites to go with them. Unit testing is a software engineering
|
|
methodology that makes it easier to build correct code. Every
|
|
method in every class should have a set of tests to verify
|
|
that they work correctly. Mono also needs a testing framework
|
|
to make it easy to write and run lots of tests.
|
|
|
|
In some classes, we might also provide standalone tests because of
|
|
some reasons such as too huge testcases, another downloading and so on.
|
|
(For example, managed XSLT has standalone test which downloads and
|
|
expands some megabytes of OASIS test suite.)
|
|
|
|
Here I list them up as long as I know. If you are going to add another
|
|
standalone tests, please add one line here. It is also recommended that
|
|
you add some notes on how to build and run tests.
|
|
|
|
<ul>
|
|
|
|
* Mono.Data/test/
|
|
* System.Data/Test, and some individual ADO.NET libraries:
|
|
there are some standalone tests. See the bottom of <a href="ado-net.html">
|
|
ADO.NET page</a> for detail.
|
|
* System.Web/Test/TestMonoWeb : see README
|
|
* System.Web.Services/Test/standalone : see README
|
|
* System.Windows.Forms/SWFTest/
|
|
* System.XML/Test/System.Xml/standalone_tests : see README
|
|
* System.XML/Test/System.Xml.Schema/standalone_tests : see README
|
|
* System.XML/System.Xml.Serialization/standalone_tests/
|
|
* System.XML/Test/System.Xml.Xsl/standalone_tests : see README
|
|
* Commons.Xml.Relaxng/Test/standalone_tests : see README
|
|
|
|
</ul>
|
|
|
|
** Getting started
|
|
|
|
If you are new to writing NUnit tests, there is a template you may use
|
|
to help get started. The file is:
|
|
|
|
<b>mcs/class/doc/TemplateTest.cs</b>
|
|
|
|
Save a copy of this file in the appropriate test subdirecty
|
|
(see below), and replace all the {text} markers with
|
|
appropriate code. Comments in the template are there to guide
|
|
you. You should also look at existing tests to see how other
|
|
people have written them.
|
|
mcs/class/corlib/Test/System.Collections/CollectionBaseTest.cs
|
|
is a small one that might help.
|
|
|
|
The directory that will contain your new file depends on the
|
|
assembly/namespace of the class for which you are creating the
|
|
tests. Under mcs/class there is a directory for each
|
|
assembly. In each assembly there is a Test directory,
|
|
e.g. mcs/class/corlib/Test. In the Test directory there are
|
|
sub-directories for each namespace in the assembly,
|
|
e.g. mcs/class/corlib/Test/Sytem. Put your new test file in
|
|
the appropriate sub-directory under Test for the class you are
|
|
testing.
|
|
|
|
Once all of that is done, you can do a 'make test' from the top mcs
|
|
directory. Your test class needs also to be listed in the
|
|
.sources file at the top of the Test directory.
|
|
|
|
* Tips on writing Unit tests.
|
|
|
|
You should look at the <a href="http://nunit.org">NUnit documentation</a>,
|
|
as it is a fantastic product, and includes fantastic documentation,
|
|
but here are some tips for those of you who are already reading
|
|
this web page.
|
|
|
|
|
|
** Provide an unique error message for Assert()
|
|
|
|
Include an unique message for each Assert() so that when the assert
|
|
fails, it is trivial to locate it in the source. Otherwise, it may be
|
|
difficult to determine which part of the test is failing. A good way
|
|
to ensure unique messages is to use something like #A01, #A02 etc.
|
|
|
|
Ok:
|
|
<pre>
|
|
|
|
AssertEquals("array match", compare[0], i1[0]);
|
|
AssertEquals("array match", compare[1], i1[1]);
|
|
AssertEquals("array match", compare[2], i1[2]);
|
|
AssertEquals("array match", compare[3], i1[3]);
|
|
</pre>
|
|
|
|
Excellent:
|
|
<pre>
|
|
AssertEquals("#A01", compare[0], i1[0]);
|
|
AssertEquals("#A02", compare[1], i1[1]);
|
|
AssertEquals("#A03", compare[2], i1[2]);
|
|
AssertEquals("#A04", compare[3], i1[3]);
|
|
</pre>
|
|
|
|
Once you used such a number in an Assert(), don't change it later on -
|
|
people might use it it identify the test in bug reports or in mailing
|
|
lists.
|
|
|
|
** Use AssertEquals() to compare things, not Assert().
|
|
|
|
Do not compare two values with Assert() - if the test fails,
|
|
people have no idea what went wrong while AssertEquals()
|
|
reports the failed value.
|
|
|
|
Ok:
|
|
<pre>
|
|
Assert ("A01", myTicks[0] == t1.Ticks);
|
|
</pre>
|
|
|
|
Excellent:
|
|
<pre>
|
|
AssertEquals ("A01", myTicks[0], t1.Ticks);
|
|
</pre>
|
|
|
|
** Test your test with the Microsoft runtime
|
|
|
|
If possible, try to run your testsuite with the Microsoft runtime on
|
|
.NET on Windows and make sure all tests in it pass. This is especially
|
|
important if you're writing a totally new testcase - without this
|
|
check you can never be sure that your testcase contains no bugs ....
|
|
|
|
Don't worry if you're writing your test on Linux, other people can
|
|
test it for you on Windows.
|
|
|
|
Sometimes you may discover that a test doesn't show the expected
|
|
result when run with the Microsoft runtime - either because there is a
|
|
bug in their runtime or something is misleading or wrong in their
|
|
documentation. In this case, please put a detailed description of the
|
|
problem to mcs/class/doc/API-notes and do also report it to the
|
|
<a href="mailing-lists.html">mailing list</a> - we'll forward this to the
|
|
Microsoft people from time to time to help them fix their documentation
|
|
and runtime.
|
|
|
|
** Unit tests.
|
|
|
|
Why do unit testing? It becomes simple to run automated tests
|
|
for the whole library. Unit tests are a safety net - you can
|
|
change part of the code and verify that you haven't broken
|
|
anything. Ideally, tests are written before the actual library
|
|
code itself. And every time a bug is discovered, a test should
|
|
be written to demonstrate the bug and its fix. Then, if
|
|
you ever reintroduce the bug, you will know immediately. For
|
|
more info, read <a
|
|
href="http://junit.sourceforge.net/doc/testinfected/testing.htm">
|
|
JUnit Test Infected: Programmers Love Writing Tests</a>.
|
|
|
|
|
|
** Getting Started
|
|
|
|
We welcome all contributions to the Class Libary Test Suite.
|
|
|
|
There is information to help you get started in CVS at
|
|
mcs/class/doc/NUnitGuidelines. Once you have written your test, please
|
|
post it to <a href="mailing-lists.html">mono-list</a>.
|
|
|
|
Someone will make sure to add the file or apply the patch as
|
|
appropriate. If you plan to be an on-going contributor and
|
|
would like to get cvs account, email <a href="mailto:[email protected]">miguel</a>.
|
|
|
|
Normally, after you send a couple of well-written new files
|
|
and/or patches to the list, you will be given cvs access.
|
|
|
|
<a name="compiler"></a>
|
|
* Compiler tests
|
|
|
|
Mono ships with three compilers: C#, VB.NET and JScript. The
|
|
tests are ran by running the makefile target `make
|
|
run-test-local' in the appropriate directory.
|
|
|
|
The C# compilation tests live in mcs/tests, and the C# error
|
|
tests live in mcs/errors.
|
|
|
|
The VB.NET compilation tests live in mcs/btests.
|
|
|
|
<a name="runtime"></a>
|
|
* Runtime Tests
|
|
|
|
These tests verify the virtual machine, to run these tests, do:
|
|
|
|
<pre>
|
|
cd mono/mono/tests
|
|
make test
|
|
</pre>
|
|
|
|
<a name="aspnet"></a>
|
|
* ASP.NET tests
|
|
|
|
XSP, the Mono ASP.NET server has tests for ASP.NET pages. It uses
|
|
<a href="http://nunitasp.sourceforge.net">NUnitAsp</a>. Right now
|
|
it only has standalone tests, ie., tests that do not need their own
|
|
global.asax or web.config files.
|
|
|
|
If you want to run them, get the xsp CVS module and install it. Then:
|
|
<pre>
|
|
cd xsp/nunit-tests
|
|
make
|
|
cd standalone
|
|
xsp
|
|
</pre>
|
|
|
|
And from another terminal:
|
|
<pre>
|
|
cd xsp/nunit-tests/standalone
|
|
nunit-console standalone-tests.dll
|
|
</pre>
|
|
|
|
<a name="ws"></a>
|
|
* Web Services tests
|
|
|
|
The Test directory for the System.Web.Services assembly contains a
|
|
standalone test suite for testing web services. It tests:
|
|
|
|
<ul>
|
|
<li>Proxy generation using the wsdl tool</li>
|
|
<li>Access to web services using the generated client proxies</li>
|
|
<li>Execution of web services in the server</li>
|
|
</ul>
|
|
|
|
This suite not only tests web services running on XSP, but it can also test
|
|
services running on other platforms and that are available in internet. This
|
|
will help track down interoperability issues.
|
|
|
|
To build the test suite, just run:
|
|
|
|
<pre>
|
|
cd mcs/class/System.Web.Services/Test/standalone
|
|
xsp --root server
|
|
</pre>
|
|
|
|
And from another terminal:
|
|
<pre>
|
|
cd mcs/class/System.Web.Services/Test/standalone
|
|
make
|
|
nunit-console testclient.dll
|
|
</pre>
|
|
|
|
This will download the wsdl documents, generate the proxies, build a dll with
|
|
the proxies, and build the nunit tests. Then you can use nunit-console or
|
|
gnunit to run the tests (the nunit dll is testclient.dll).
|
|
|
|
Read the README file in mcs/class/System.Web.Services/Test/standalone for
|
|
more info.
|