[libc++] Add documentation for setting up new CI jobs

libcxx/docs/AddingNewCIJobs.rst

@@ -0,0 +1,67 @@
+.. _AddingNewCIJobs:
+
+==================
+Adding New CI Jobs
+==================
+
+.. contents::
+  :local:
+
+Adding The Job
+==============
+
+libc++ uses Buildkite for running its CI. Setting up new CI jobs is easy, and
+these jobs can run either on our existing infrastructure, or on your own.
+
+If you need to run the job on your own machines, please follow the
+`Buildkite guide <https://buildkite.com/docs/agent/v3>`_ to setup your
+own agents. Make sure you tag your agents in a way that you'll be able
+to recognize them when defining your job below. Finally, in order for the
+agent to register itself to Buildkite, it will need a ``BUILDKITE_AGENT_TOKEN``.
+Please contact a maintainer to get your token.
+
+Then, simply add a job to the Buildkite pipeline by editing ``libcxx/utils/ci/buildkite-pipeline.yml``.
+Take a look at how the surrounding jobs are defined and do something similar.
+An example of a job definition is:
+
+.. code-block:: yaml
+
+  - label: "C++11"
+    command: "libcxx/utils/ci/run-buildbot generic-cxx11"
+    artifact_paths:
+      - "**/test-results.xml"
+    agents:
+      queue: "libcxx-builders"
+    retry:
+      automatic:
+        - exit_status: -1  # Agent was lost
+          limit: 2
+
+If you've created your own agents, you should provide the tag that you used
+when creating them in the ``queue`` entry -- this will instruct Buildkite to
+run that job only on agents that have that tag.
+
+We try to keep the pipeline definition file as simple as possible, and to
+keep any script used for CI inside ``libcxx/utils/ci``. This ensures that
+it's possible to reproduce CI issues locally with ease, understanding of
+course that some setups may require access to special hardware that is not
+available.
+
+Testing Your New Job
+====================
+
+Testing your new job is easy -- once your agent is set up (if any), just open
+a code review and the libc++ CI pipeline will run, including any changes you
+might have made to the pipeline definition itself.
+
+Service Level Agreement
+=======================
+
+To keep the libc++ CI useful for everyone, we aim for a quick turnaround time
+for all CI jobs. This allows the overall pipeline to finish in a reasonable
+amount of time, which is important because it directly affects our development
+velocity. We also try to make sure that jobs run on reliable infrastructure in
+order to avoid flaky failures, which reduce the value of CI for everyone.
+
+We may be reluctant to add and support CI jobs that take a long time to finish
+or that are too flaky.

libcxx/docs/index.rst

@@ -43,8 +43,10 @@ Getting Started with libc++
 .. toctree::
     :hidden:
 
+    AddingNewCIJobs
     FeatureTestMacroTable
 
+
 Current Status
 --------------
 
@@ -174,10 +176,9 @@ Design Documents
 Build Bots and Test Coverage
 ----------------------------
 
-* `LLVM Buildbot Builders <http://lab.llvm.org:8011/console>`_
-* `Apple Jenkins Builders <http://lab.llvm.org:8080/green/view/Libcxx/>`_
-* `Windows Appveyor Builders <https://ci.appveyor.com/project/llvm-mirror/libcxx>`_
-* `Code Coverage Results <http://efcs.ca/libcxx-coverage>`_
+* `Buildkite CI pipeline <https://buildkite.com/llvm-project/libcxx-ci>`_
+* `LLVM Buildbot Builders <http://lab.llvm.org:8011>`_
+* :ref:`Adding New CI Jobs <AddingNewCIJobs>`
 
 Getting Involved
 ================