mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-12-25 17:43:44 +00:00
This is Tinderbox. See <http://www.mozilla.org/tinderbox.html>. ========== DISCLAIMER ========== This is not very well packaged code. It's not packaged at all. Don't come here expecting something you plop in a directory, twiddle a few things, and you're off and using it. Much work has to be done to get there. We'd like to get there, but it wasn't clear when that would be, and so we decided to let people see it first. Don't believe for a minute that you can use this stuff without first understanding most of the code. ============ DEPENDENCIES ============ To use tinderbox, you must first have bonsai up and running. See <http://www.mozilla.org/bonsai.html>. Be warned now that bonsai is not easily installed. ==================================== What's What in the Tinderbox sources: ==================================== This is a rough first pass at cataloging and documenting the Tinderbox sources. Many hands have been in this code over the years, and it has accreted wildly. There is probably quite a lot of dead code in here. PROGRAMS ======== handlemail.pl This is the mail deliverty agent (MDA) for tinderbox, the local message transfer agent (MTA) calls this program to deliver the tinderbox mail into the system. It is a wrapper for processbuild.pl. processbuild.pl Update the "$tbx{tree}/build.dat" database as new mail comes in then run "./buildwho.pl $tree" and "./showbuilds.cgi" (to build static versions of tinderbox data. buildwho.pl Update the who.dat file with the list of authors who checked in in the last two days. This is run from processbuild.pl, always, and from showbuild.cgi when 'rebuildguilty' is clicked Conceptually, the three programs, handlemail.pl, processbuild.pl, and buildwho.pl, make up the MDA for tinderbox. showbuilds.cgi Create the Tinderbox web page. showlog.cgi Show a build log (brief and full) update the brief_log if necessary. Requires the ep_$form{errorparser}.pl error parser. ep_unix.pl Knows how to parse Unix build error logs. Used by processbuild.pl There needs to be one ep_* file for each distinct parsing algorithm used, typically this is per platform. This file defines the regular expressions which locate error lines: has_error() has_warning() and the function has_errorline() which parses the line into the global variables: $error_file, $error_file_ref, $error_line, $error_guess, admintree.cgi Displays the admin cgi which depends on: $message_of_day, $ignore_builds doadmin.cgi Actually do the work to admin a tinderbox tree (change message of the day, turn off displays for a channel) clean.pl Run `find . -name \"*.gz\" -mtime +7 -print ` and unlink those files. Does not appear to be run from other tools. It is a good candidate for a cron job. OPTIONS to showbuilds.cgi ========================= Options to showbuilds are specified in the URL and are undocumented elsewhere. If called with no 'tree' option display the possible build trees to pick from. The 'tree' picks the build to display. An additional tree can be specified with 'tree2'. Interesting visual params are: current state monitoring mode: express=1 or panel=1 text mode state monitoring: quickparse=1 These modes do not show on my browser: flash=1 rdf=1 static=1 These are self explanatory: legend=0; norules=0; hours=n; EMAIL FORMAT ============ Each tinderbox client mails status updates to the tinderbox server. These mails contain special tinderbox data lines describing the progress of the build it may also contain the log of the build process. The email to the tinderbox server looks like: tinderbox: tree: Mozilla tinderbox: builddate: 900002087 tinderbox: status: building tinderbox: build: IRIX 6.3 Depend tinderbox: errorparser: unix If binaryurl is set then a "D" link shows up in that builds status panel. This link points to an ftp url where you can fetch the build from the ftp server that was specified. # NOT USED tinderbox: buildfamily: unix DATA STRUCTURES IN showbuilds.cgi ================================= load_buildlog() creates build_list a list of hash refernces of this type $buildrec = { mailtime => $mailtime, buildtime => $buildtime, buildname => ($tree2 ne '' ? $t->{name} . ' ' : '' ) . $buildname, errorparser => $errorparser, buildstatus => $buildstatus, logfile => $logfile, binaryname => $binaryname, td => $t }; the $buildrec->{rowspan} variable holds the number of rows that the build should occupy on the table and is not stored in the build.dat file. These are other add ons to the data structure $buildrec->{hasnote}=1; $buildrec->{noteid} = (0+@note_array); commonly buildrec's are accessed through: $build_table->[$build_time][$build_name] = $buildrec; The list of users who updated this build is stored as: $who_list->[$checkin_time]->{$author} = 1; There are numerous duplicate data structures which hold partial information: hashes which hold all indices: $build_name_index->{$br->{buildname}} = 1; $build_time_index->{$br->{buildtime}} = 1; other access into $build_table: $build_names->[$i] = $n; $build_time_times->[$i] = $n; loadquickparseinfo creates these references $build->{$buildname} = $buildstatus; $times->{$buildname} = $buildtime; DATA FILES ========== These files are used to store data structures. They are a persistent store of data between executions of the same program and they pass the data between cooperating program. This data is often databases with rows separated by "\n" and columns by '|'. $tree/${logfile} $tree/${logfile}.brief.html $tree/ignorebuilds.pl $tree/scrapebuilds.pl $tree/rules.pl $tree/status.pl $tree/sheriff.pl $tree/notes.txt $tree/treedata.pl $tree/who.dat $treename/build.dat The logfile that the tinderbox client sent stored in gziped format. The filename is ${tree}/$builddate.$$.gz so its quite random and does not depend on the clients$build string and multiple logs are kept for each build, one for each mail message sent. The brief.html file is a cache of the error log summary for this log file and is recreated when the logfile gets updated. ignorebuilds.pl is a file which specifies builds that should not be performed. It is valid perl code which sets the hash reference $ignore_builds, each key is a tree name. scrapebuilds.pl: same as ignorebuilds.pl, but selects a build to be "scraped" for data, with a token in the form TinderboxPrint:aaa,bbb,... status.pl stores the tree-specific status. Its contents looks like: $status_message = 'message'; 1; rules.pl stores the tree rules message. Its contents looks like: $rules_message = 'message'; 1; sheriff.pl stores the current sheriff. Its contents looks like: $current_sheriff = 'sheriff'; 1; notes.txt store the database of notes: $buildtime|$buildname||$author|$time_now|$note treedata.pl stores the cvs information pertaining to this tree and looks like this: $cvs_module='$modulename'; $cvs_branch='$branchname'; $cvs_root='$repository'; who.dat file has lines like: $checkin_time|$author This gets stored in the data structure, where checkin_time gets fudged up so that is is a time already in the build_table: $who_list->[$checkin_time]->{$author} = 1; build.dat stores the build results table. It is a flat file representation of $build_table build.dat is a database file each row is a build and has pipe separated columns: 1) the time stamp of the tinderbox server 2) time stamp of the build machine 3) the official build name (should include build machine name) ( note: that 2 & 3 together uniquely identify the build and all relevant build data) 4) the architecture dependent error parser to use on the log files 5) status of the build (success|busted|building|testfailed) 6) The log file for this build (if completed) 7) the name of the binary (if any) that came from the build Other Files ==================== 1afi003r.gif The "flames" animation used by showbuilds.cgi Empty.html Document used for an empty frame by ??? addimage.cgi The form that lets you add a new image to the list of images that Tinderbox picks from randomly. addnote.cgi Add a note to a build log. admintree.cgi Lets you perform various admin tasks on a Tinderbox tree. This just prompts for a password and posts to doadmin.cgi. clean.pl ??? copydata.pl ??? doadmin.cgi Actually do the work to admin a tinderbox tree ep_mac.pl Knows how to parse Mac build error logs. Used by ??? ep_unix.pl Knows how to parse Unix build error logs. Used by ??? ep_windows.pl Knows how to parse Windows build error logs. Used by ??? faq.html Wildly out of date. fixupimages.pl ??? tbglobals.pl ??? imagelog.pl ??? index.html ??? reledanim.gif ??? showimages.cgi Show all the images in the Tinderbox list. Password-protected. star.gif The "star" image used to annotate builds by ???