Bug 1720464 - land NSS 8f41147c2192 UPGRADE_NSS_RELEASE, r=beurdouche

```
2021-07-22  Benjamin Beurdouche  <bbeurdouche@mozilla.com>

	* doc/rst/index.rst:
	Display warning on the new NSS documentation
	[8f41147c2192] [tip]

2021-07-20  Robert Relyea  <rrelyea@redhat.com>

	* lib/softoken/sdb.c:
	Bug 1721476 sqlite 3.34 changed it's open semantics, causing nss
	failures.

	https://sqlite.org/forum/info/42cf8e985bb051a2

	sqlite is now permissive on opening a readonly file even if you ask
	for the file to be opened R/W.

	normally sqlite is very conservative in changing it's underlying
	semantics, but evidently they chose convience over compatibility.
	NSS now needs to check the file permissions itself to preserve nss
	semantics.

	[f2d34a957599]

2021-07-15  Robert Relyea  <rrelyea@redhat.com>

	* tests/common/init.sh, tests/common/parsegtestreport.sed,
	tests/common/parsegtestreport.sh, tests/gtests/gtests.sh,
	tests/ssl_gtests/ssl_gtests.sh:
	Bug 1720230 Gtest update changed the gtest reports, losing gtest
	details in all.sh reports.

	This patch includes the updated .sed script, and an experiment using
	bash instead to see how hard it would be to make a more robust
	parser.

	The robust parser generates identical output as sed, but takes about
	30x longer, so instead of subsecond operations, it takes almost half
	a minute. With that result, I think we can stay with sed and
	continue to update when we get new versions of gtests. (sigh).

	time cat report.xml.0 | sed -f parsegtestreport.sed > r1

	real 0m0.710s user 0m0.705s sys 0m0.008s

	time cat report.xml.0 | sh parsegtestreport.sh > r2

	real 0m25.066s user 0m17.759s sys 0m9.506s [rrelyea@localhost
	common]$ diff r1 r2

	updated: with review comments from Martin and move the report
	parsing to the common code so it can be shared with both ssl_gtests
	and gtests shell scripts.

	[f12856d5d2c2]

2021-07-13  Robert Relyea  <rrelyea@redhat.com>

	* gtests/softoken_gtest/softoken_dh_vectors.h, lib/softoken/pkcs11c.c,
	lib/softoken/pkcs11i.h, lib/softoken/pkcs11u.c,
	lib/softoken/sftkdhverify.c:
	Bug 1720228 NSS incorrectly accepting 1536 bit DH primes in FIPS
	mode

	When NSS is in FIPS mode, it should reject all primes smaller than
	2048. The ike 1536 prime is in the accepted primes table. In FIPS
	mode it should be rejected.

	[d2ec946e601a]

2021-07-15  Robert Relyea  <rrelyea@redhat.com>

	* cmd/manifest.mn, cmd/sdbthreadtst/Makefile,
	cmd/sdbthreadtst/manifest.mn, cmd/sdbthreadtst/sdbthreadtst.c,
	cmd/sdbthreadtst/sdbthreadtst.gyp, lib/softoken/sdb.c,
	lib/softoken/sftkdb.c, nss.gyp, tests/dbtests/dbtests.sh:
	Bug 1720232 SQLite calls could timeout in starvation situations.

	Some of our servers could cause random failures when trying to
	generate many key pairs from multiple threads. This is caused
	because some threads would starve long enough for them to give up on
	getting a begin transaction on sqlite. sqlite only allows one
	transaction at a time.

	Also, there were some bugs in error handling of the broken
	transaction case where NSS would try to cancel a transation after
	the begin failed (most cases were correct, but one case in
	particular was problematic).

	[b54b0d41e51b]

2021-07-13  Robert Relyea  <rrelyea@redhat.com>

	* lib/pk11wrap/pk11cxt.c, lib/pk11wrap/pk11hpke.c,
	lib/softoken/kbkdf.c, lib/softoken/sftkhmac.c,
	lib/softoken/sftkike.c:
	Bug 1720225 Coverity/cpp scanner errors found in nss 3.67

	A number of coverity/scanner issues were found in the kdf code which
	was added in nss 3.44 and the fixes never upstreamed, as well as
	coverity/scanner errors in nss 3.66. Not all errors were fixed,
	those errors which were determined to be false positives were just
	recorded. No attempt has been made to fix coverity/scanner errors in
	gtests.

	[d1b9709d8861]
```

Differential Revision: https://phabricator.services.mozilla.com/D120624
This commit is contained in:
Benjamin Beurdouche 2021-07-23 09:23:50 +00:00
parent 43ad6da6da
commit dde8b5dd22
27 changed files with 518 additions and 99 deletions

View File

@ -1 +1 @@
b1eac8c86e99
8f41147c2192

View File

@ -65,6 +65,7 @@ NSS_SRCDIRS = \
pwdecrypt \
rsaperf \
rsapoptst \
sdbthreadtst \
sdrtest \
selfserv \
signtool \

View File

@ -0,0 +1,48 @@
#! gmake
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#######################################################################
# (1) Include initial platform-independent assignments (MANDATORY). #
#######################################################################
include manifest.mn
#######################################################################
# (2) Include "global" configuration information. (OPTIONAL) #
#######################################################################
include $(CORE_DEPTH)/coreconf/config.mk
#######################################################################
# (3) Include "component" configuration information. (OPTIONAL) #
#######################################################################
#######################################################################
# (4) Include "local" platform-dependent assignments (OPTIONAL). #
#######################################################################
include ../platlibs.mk
#######################################################################
# (5) Execute "global" rules. (OPTIONAL) #
#######################################################################
include $(CORE_DEPTH)/coreconf/rules.mk
#######################################################################
# (6) Execute "component" rules. (OPTIONAL) #
#######################################################################
#######################################################################
# (7) Execute "local" rules. (OPTIONAL). #
#######################################################################
include ../platrules.mk

View File

@ -0,0 +1,22 @@
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
CORE_DEPTH = ../..
DEFINES += -DNSPR20
# MODULE public and private header directories are implicitly REQUIRED.
MODULE = nss
CSRCS = \
sdbthreadtst.c \
$(NULL)
# The MODULE is always implicitly required.
# Listing it here in REQUIRES makes it appear twice in the cc command line.
PROGRAM = sdbthreadtst
# USE_STATIC_LIBS = 1

View File

@ -0,0 +1,213 @@
#if defined(XP_UNIX)
#include <unistd.h>
#endif
#include <stdio.h>
#include <nss.h>
#include <prtypes.h>
#include <prerr.h>
#include <prerror.h>
#include <prthread.h>
#include <pk11pub.h>
#include <keyhi.h>
#define MAX_THREAD_COUNT 100
/* globals */
int THREAD_COUNT = 30;
int FAILED = 0;
int ERROR = 0;
int LOOP_COUNT = 100;
int KEY_SIZE = 3072;
int STACK_SIZE = 0;
int VERBOSE = 0;
char *NSSDIR = ".";
PRBool ISTOKEN = PR_TRUE;
CK_MECHANISM_TYPE MECHANISM = CKM_RSA_PKCS_KEY_PAIR_GEN;
void
usage(char *prog, char *error)
{
if (error) {
fprintf(stderr, "Bad Arguments: %s", error);
}
fprintf(stderr, "usage: %s [-l loop_count] [-t thread_count] "
"[-k key_size] [-s stack_size] [-d nss_dir] [-e] [-v] [-h]\n",
prog);
fprintf(stderr, " loop_count -- "
"number of keys to generate on each thread (default=%d)\n",
LOOP_COUNT);
fprintf(stderr, " thread_count -- "
"number of of concurrent threads to run (def=%d,max=%d)\n",
THREAD_COUNT, MAX_THREAD_COUNT);
fprintf(stderr, " key_size -- "
"rsa key size in bits (default=%d)\n",
KEY_SIZE);
fprintf(stderr, " stack_size -- "
"thread stack size in bytes, 0=optimal (default=%d)\n",
STACK_SIZE);
fprintf(stderr, " nss_dir -- "
"location of the nss directory (default=%s)\n",
NSSDIR);
fprintf(stderr, " -e use session keys rather than token keys\n");
fprintf(stderr, " -v verbose, print progress indicators\n");
fprintf(stderr, " -h print this message\n");
exit(2);
}
void
create_key_loop(void *arg)
{
int i;
PK11SlotInfo *slot = PK11_GetInternalKeySlot();
PK11RSAGenParams param;
int threadnumber = *(int *)arg;
int failures = 0;
int progress = 5;
PRIntervalTime epoch = PR_IntervalNow();
param.keySizeInBits = KEY_SIZE;
param.pe = 0x10001L;
printf(" - thread %d starting\n", threadnumber);
progress = 30 / THREAD_COUNT;
if (progress < 2)
progress = 2;
for (i = 0; i < LOOP_COUNT; i++) {
SECKEYPrivateKey *privKey;
SECKEYPublicKey *pubKey;
privKey = PK11_GenerateKeyPair(slot, MECHANISM, &param, &pubKey,
ISTOKEN, PR_TRUE, NULL);
if (privKey == NULL) {
fprintf(stderr,
"keypair gen in thread %d failed %s\n", threadnumber,
PORT_ErrorToString(PORT_GetError()));
FAILED++;
failures++;
}
if (VERBOSE && (i % progress) == 0) {
PRIntervalTime current = PR_IntervalNow();
PRIntervalTime interval = current - epoch;
int seconds = (interval / PR_TicksPerSecond());
int mseconds = ((interval * 1000) / PR_TicksPerSecond()) - (seconds * 1000);
epoch = current;
printf(" - thread %d @ %d iterations %d.%03d sec\n", threadnumber,
i, seconds, mseconds);
}
if (ISTOKEN && privKey) {
SECKEY_DestroyPublicKey(pubKey);
SECKEY_DestroyPrivateKey(privKey);
}
}
PK11_FreeSlot(slot);
printf(" * thread %d ending with %d failures\n", threadnumber, failures);
return;
}
int
main(int argc, char **argv)
{
PRThread *thread[MAX_THREAD_COUNT];
int threadnumber[MAX_THREAD_COUNT];
int i;
PRStatus status;
SECStatus rv;
char *prog = *argv++;
char buf[2048];
char *arg;
while ((arg = *argv++) != NULL) {
if (*arg == '-') {
switch (arg[1]) {
case 'l':
if (*argv == NULL)
usage(prog, "missing loop count");
LOOP_COUNT = atoi(*argv++);
break;
case 'k':
if (*argv == NULL)
usage(prog, "missing key size");
KEY_SIZE = atoi(*argv++);
break;
case 's':
if (*argv == NULL)
usage(prog, "missing stack size");
STACK_SIZE = atoi(*argv++);
break;
case 't':
if (*argv == NULL)
usage(prog, "missing thread count");
THREAD_COUNT = atoi(*argv++);
if (THREAD_COUNT > MAX_THREAD_COUNT) {
usage(prog, "max thread count exceeded");
}
break;
case 'v':
VERBOSE = 1;
break;
case 'd':
if (*argv == NULL)
usage(prog, "missing directory");
NSSDIR = *argv++;
break;
case 'e':
ISTOKEN = PR_FALSE;
break;
case 'h':
usage(prog, NULL);
break;
default:
sprintf(buf, "unknown option %c", arg[1]);
usage(prog, buf);
}
} else {
sprintf(buf, "unknown argument %s", arg);
usage(prog, buf);
}
}
/* initialize NSS */
rv = NSS_InitReadWrite(NSSDIR);
if (rv != SECSuccess) {
fprintf(stderr,
"NSS_InitReadWrite(%s) failed(%s)\n", NSSDIR,
PORT_ErrorToString(PORT_GetError()));
exit(2);
}
/* need to initialize the database here if it's not already */
printf("creating %d threads\n", THREAD_COUNT);
for (i = 0; i < THREAD_COUNT; i++) {
threadnumber[i] = i;
thread[i] = PR_CreateThread(PR_USER_THREAD, create_key_loop,
&threadnumber[i], PR_PRIORITY_NORMAL,
PR_GLOBAL_THREAD,
PR_JOINABLE_THREAD, STACK_SIZE);
if (thread[i] == NULL) {
ERROR++;
fprintf(stderr,
"PR_CreateThread failed iteration %d, %s\n", i,
PORT_ErrorToString(PORT_GetError()));
}
}
printf("waiting on %d threads\n", THREAD_COUNT);
for (i = 0; i < THREAD_COUNT; i++) {
if (thread[i] == NULL) {
continue;
}
status = PR_JoinThread(thread[i]);
if (status != PR_SUCCESS) {
ERROR++;
fprintf(stderr,
"PR_CreateThread filed iteration %d, %s]n", i,
PORT_ErrorToString(PORT_GetError()));
}
}
printf("%d failures and %d errors found\n", FAILED, ERROR);
/* clean up */
NSS_Shutdown();
if (FAILED) {
exit(1);
}
if (ERROR) {
exit(2);
}
exit(0);
}

View File

@ -0,0 +1,29 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
{
'includes': [
'../../coreconf/config.gypi',
'../../cmd/platlibs.gypi'
],
'targets': [
{
'target_name': 'sdbthreadtst',
'type': 'executable',
'sources': [
'sdbthreadtst.c'
],
'dependencies': [
'<(DEPTH)/exports.gyp:nss_exports'
]
}
],
'target_defaults': {
'defines': [
'NSPR20'
]
},
'variables': {
'module': 'nss'
}
}

View File

@ -10,3 +10,4 @@
*/
#error "Do not include this header file."

View File

@ -12,6 +12,10 @@ Network Security Services
introduction_to_network_security_services/index.rst
More documentation <more_docs>
.. warning::
This NSS documentation was just imported from our legacy MDN repository. It currently is very deprecated and likely incorrect or broken in many places.
`Documentation <#documentation>`__
----------------------------------

View File

@ -2872,7 +2872,7 @@ static const DhTestVector DH_TEST_VECTORS[] = {
{siBuffer, (unsigned char *)g2, sizeof(g2)},
{siBuffer, NULL, 0},
{siBuffer, NULL, 0},
IKE_APPROVED,
SAFE_PRIME,
CLASS_1536},
{"IKE 2048",
{siBuffer, (unsigned char *)prime_ike_2048, sizeof(prime_ike_2048)},
@ -2952,7 +2952,7 @@ static const DhTestVector DH_TEST_VECTORS[] = {
{siBuffer, (unsigned char *)sub2_prime_ike_1536,
sizeof(sub2_prime_ike_1536)},
{siBuffer, NULL, 0},
IKE_APPROVED,
SAFE_PRIME,
CLASS_1536},
{"IKE 2048 with subprime",
{siBuffer, (unsigned char *)prime_ike_2048, sizeof(prime_ike_2048)},

View File

@ -382,7 +382,7 @@ pk11_CreateNewContextInSlot(CK_MECHANISM_TYPE type,
* of the connection.*/
context->fortezzaHack = PR_FALSE;
if (type == CKM_SKIPJACK_CBC64) {
if (symKey->origin == PK11_OriginFortezzaHack) {
if (symKey && (symKey->origin == PK11_OriginFortezzaHack)) {
context->fortezzaHack = PR_TRUE;
}
}

View File

@ -1164,8 +1164,6 @@ PK11_HPKE_Seal(HpkeContext *cx, const SECItem *aad, const SECItem *pt,
unsigned char tagBuf[HASH_LENGTH_MAX];
size_t tagLen;
unsigned int fixedBits;
PORT_Assert(cx->baseNonce->len == sizeof(ivOut));
PORT_Memcpy(ivOut, cx->baseNonce->data, cx->baseNonce->len);
/* aad may be NULL, PT may be zero-length but not NULL. */
if (!cx || !cx->aeadContext ||
@ -1176,6 +1174,9 @@ PK11_HPKE_Seal(HpkeContext *cx, const SECItem *aad, const SECItem *pt,
return SECFailure;
}
PORT_Assert(cx->baseNonce->len == sizeof(ivOut));
PORT_Memcpy(ivOut, cx->baseNonce->data, cx->baseNonce->len);
tagLen = cx->aeadParams->tagLen;
maxOut = pt->len + tagLen;
fixedBits = (cx->baseNonce->len - 8) * 8;

View File

@ -613,6 +613,10 @@ kbkdf_CreateKey(CK_MECHANISM_TYPE kdf_mech, CK_SESSION_HANDLE hSession, CK_DERIV
PR_ASSERT(derived_key != NULL);
PR_ASSERT(derived_key->phKey != NULL);
if (slot == NULL) {
return CKR_SESSION_HANDLE_INVALID;
}
/* Create the new key object for this additional derived key. */
key = sftk_NewObject(slot);
if (key == NULL) {
@ -678,7 +682,9 @@ done:
sftk_FreeObject(key);
/* Doesn't do anything. */
sftk_FreeSession(session);
if (session) {
sftk_FreeSession(session);
}
return ret;
}

View File

@ -5193,7 +5193,7 @@ sftk_PairwiseConsistencyCheck(CK_SESSION_HANDLE hSession, SFTKSlot *slot,
/* subprime not supplied, In this case look it up.
* This only works with approved primes, but in FIPS mode
* that's the only kine of prime that will get here */
subPrimePtr = sftk_VerifyDH_Prime(&prime);
subPrimePtr = sftk_VerifyDH_Prime(&prime, isFIPS);
if (subPrimePtr == NULL) {
crv = CKR_GENERAL_ERROR;
goto done;
@ -8351,7 +8351,7 @@ NSC_DeriveKey(CK_SESSION_HANDLE hSession,
/* if the prime is an approved prime, we can skip all the other
* checks. */
subPrime = sftk_VerifyDH_Prime(&dhPrime);
subPrime = sftk_VerifyDH_Prime(&dhPrime, isFIPS);
if (subPrime == NULL) {
SECItem dhSubPrime;
/* If the caller set the subprime value, it means that

View File

@ -946,7 +946,7 @@ char **NSC_ModuleDBFunc(unsigned long function, char *parameters, void *args);
/* dh verify functions */
/* verify that dhPrime matches one of our known primes, and if so return
* it's subprime value */
const SECItem *sftk_VerifyDH_Prime(SECItem *dhPrime);
const SECItem *sftk_VerifyDH_Prime(SECItem *dhPrime, PRBool isFIPS);
/* check if dhSubPrime claims dhPrime is a safe prime. */
SECStatus sftk_IsSafePrime(SECItem *dhPrime, SECItem *dhSubPrime, PRBool *isSafe);
/* map an operation Attribute to a Mechanism flag */

View File

@ -2312,7 +2312,7 @@ sftk_handleSpecial(SFTKSlot *slot, CK_MECHANISM *mech,
if (crv != CKR_OK) {
return PR_FALSE;
}
dhSubPrime = sftk_VerifyDH_Prime(&dhPrime);
dhSubPrime = sftk_VerifyDH_Prime(&dhPrime, PR_TRUE);
SECITEM_ZfreeItem(&dhPrime, PR_FALSE);
return (dhSubPrime) ? PR_TRUE : PR_FALSE;
}

View File

@ -82,12 +82,12 @@ typedef enum {
* total wait time for automatic operations:
* 1 second (SDB_SQLITE_BUSY_TIMEOUT/1000).
* total wait time for manual operations:
* (1 second + 5 seconds) * 10 = 60 seconds.
* (1 second + SDB_BUSY_RETRY_TIME) * 30 = 30 seconds.
* (SDB_SQLITE_BUSY_TIMEOUT/1000 + SDB_BUSY_RETRY_TIME)*SDB_MAX_BUSY_RETRIES
*/
#define SDB_SQLITE_BUSY_TIMEOUT 1000 /* milliseconds */
#define SDB_BUSY_RETRY_TIME 5 /* seconds */
#define SDB_MAX_BUSY_RETRIES 10
#define SDB_BUSY_RETRY_TIME 5 /* 'ticks', varies by platforms */
#define SDB_MAX_BUSY_RETRIES 30
/*
* known attributes
@ -690,6 +690,11 @@ sdb_openDB(const char *name, sqlite3 **sqlDB, int flags)
openFlags = SQLITE_OPEN_READONLY;
} else {
openFlags = SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE;
/* sqlite 3.34 seem to incorrectly open readwrite.
* when the file is readonly. Explicitly reject that issue here */
if ((_NSSUTIL_Access(name, PR_ACCESS_EXISTS) == PR_SUCCESS) && (_NSSUTIL_Access(name, PR_ACCESS_WRITE_OK) != PR_SUCCESS)) {
return SQLITE_READONLY;
}
}
/* Requires SQLite 3.5.0 or newer. */
@ -1001,6 +1006,7 @@ sdb_GetValidAttributeValueNoLock(SDB *sdb, CK_OBJECT_HANDLE object_id,
found = 1;
}
} while (!sdb_done(sqlerr, &retry));
sqlite3_reset(stmt);
sqlite3_finalize(stmt);
stmt = NULL;
@ -1524,6 +1530,8 @@ sdb_Begin(SDB *sdb)
if (sqlerr == SQLITE_BUSY) {
PR_Sleep(SDB_BUSY_RETRY_TIME);
}
/* don't retry BEGIN transaction*/
retry = 0;
} while (!sdb_done(sqlerr, &retry));
if (stmt) {
@ -2261,6 +2269,7 @@ sdb_init(char *dbname, char *table, sdbDataType type, int *inUpdate,
}
}
} while (!sdb_done(sqlerr, &retry));
if (sqlerr != SQLITE_DONE) {
goto loser;
}

View File

@ -1526,7 +1526,7 @@ sftkdb_DestroyObject(SFTKDBHandle *handle, CK_OBJECT_HANDLE objectID,
crv = (*db->sdb_Begin)(db);
if (crv != CKR_OK) {
goto loser;
return crv;
}
crv = (*db->sdb_DestroyObject)(db, objectID);
if (crv != CKR_OK) {
@ -2461,7 +2461,7 @@ sftkdb_Update(SFTKDBHandle *handle, SECItem *key)
*/
crv = (*handle->db->sdb_Begin)(handle->db);
if (crv != CKR_OK) {
goto loser;
return crv;
}
inTransaction = PR_TRUE;

View File

@ -1171,11 +1171,15 @@ static const SECItem subprime_tls_8192 =
* verify that dhPrime matches one of our known primes
*/
const SECItem *
sftk_VerifyDH_Prime(SECItem *dhPrime)
sftk_VerifyDH_Prime(SECItem *dhPrime, PRBool isFIPS)
{
/* use the length to decide which primes to check */
switch (dhPrime->len) {
case 1536 / PR_BITS_PER_BYTE:
/* don't accept 1536 bit primes in FIPS mode */
if (isFIPS) {
break;
}
if (PORT_Memcmp(dhPrime->data, prime_ike_1536,
sizeof(prime_ike_1536)) == 0) {
return &subprime_ike_1536;

View File

@ -234,7 +234,9 @@ sftk_MAC_Init(sftk_MACCtx *ctx, CK_MECHANISM_TYPE mech, SFTKObject *key)
keyval->attrib.ulValueLen, isFIPS);
done:
sftk_FreeAttribute(keyval);
if (keyval) {
sftk_FreeAttribute(keyval);
}
return ret;
}

View File

@ -1411,7 +1411,6 @@ sftk_fips_IKE_PowerUpSelfTests(void)
(outKeySize != sizeof(ike_known_sha256_prf_plus)) ||
(PORT_Memcmp(outKeyData, ike_known_sha256_prf_plus,
sizeof(ike_known_sha256_prf_plus)) != 0)) {
PORT_ZFree(outKeyData, outKeySize);
PORT_SetError(SEC_ERROR_LIBRARY_FAILURE);
return SECFailure;
}

View File

@ -189,6 +189,7 @@
'cmd/pp/pp.gyp:pp',
'cmd/rsaperf/rsaperf.gyp:rsaperf',
'cmd/rsapoptst/rsapoptst.gyp:rsapoptst',
'cmd/sdbthreadtst/sdbthreadtst.gyp:sdbthreadtst',
'cmd/sdrtest/sdrtest.gyp:sdrtest',
'cmd/selfserv/selfserv.gyp:selfserv',
'cmd/shlibsign/mangle/mangle.gyp:mangle',

View File

@ -253,6 +253,72 @@ if [ -z "${INIT_SOURCED}" -o "${INIT_SOURCED}" != "TRUE" ]; then
HTML_UNKNOWN='</TD><TD>Unknown</TD><TR>'
TABLE_ARGS=
gtest_parse_report_helper()
{
# Check XML reports for normal test runs and failures.
local successes=$(gtest_parse_report_xpath "//testcase[@status='run'][count(*)=0]" "$@" )
local failures=$(gtest_parse_report_xpath "//failure/.." "$@" )
# Print all tests that succeeded.
while read result name; do
html_passed_ignore_core "$name"
done <<< "$successes"
# Print failing tests.
if [ -n "$failures" ]; then
printf "\nFAILURES:\n=========\n"
while read result name; do
html_failed_ignore_core "$name"
done <<< "$failures"
printf "\n"
fi
}
# This legacy report parser can't actually detect failures. It always relied
# on the binary's exit code. Print the tests we ran to keep the old behavior.
gtest_parse_report_legacy()
{
while read result name && [ -n "$name" ]; do
if [ "$result" = "notrun" ]; then
echo "$name" SKIPPED
elif [ "$result" = "run" ]; then
html_passed_ignore_core "$name"
else
html_failed_ignore_core "$name"
fi
done <<< "$(sed -f "${COMMON}/parsegtestreport.sed" "$@" )"
# here's how we would use bash if it wasn't so slow
# done <<< "$(sh "${COMMON}/parsegtestreport.sh" "$@" )"
}
gtest_parse_report_xpath()
{
# Query the XML report with the given XPath pattern.
xpath="$1"
shift
xmllint --xpath "${xpath}" "$@" 2>/dev/null | \
# Insert newlines to help sed.
sed $'s/<testcase/\\\n<testcase/g' | \
# Use sed to parse the report.
sed -f "${COMMON}/parsegtestreport.sed"
# here's how we would use bash if it wasn't so slow
#sh "${COMMON}/parsegtestreport.sh"
}
gtest_parse_report()
{
if type xmllint &>/dev/null; then
echo "DEBUG: Using xmllint to parse GTest XML report(s)"
gtest_parse_report_helper "$@"
else
echo "DEBUG: Falling back to legacy XML report parsing using only sed"
gtest_parse_report_legacy "$@"
fi
}
#directory name init
SCRIPTNAME=init.sh

View File

@ -1,8 +1,12 @@
/\<testcase/{
s/^.* name="\([^"]*\)" value_param="\([^"]*\)" status="\([^"]*\)" time="[^"]*" classname="\([^"]*\)".*$/\3 '\4: \1 \2'/
s/^.* name="\([^"]*\)" value_param="\([^"]*\)" status="\([^"]*\)" time="[^"]*" classname="\([^"]*\).*$/\3 '\4: \1 \2'/
t end
s/^.* name="\([^"]*\)" status="\([^"]*\)" time="[^"]*" classname="\([^"]*\)".*$/\2 '\3: \1'/
t end
s/^.* name="\([^"]*\)" value_param="\([^"]*\)" status="\([^"]*\)" result="[^"]*" time="[^"]*" timestamp="[^"]*" classname="\([^"]*\)".*$/\3 '\4: \1 \2'/
t end
s/^.* name="\([^"]*\)" status="\([^"]*\)" result="[^"]*" time="[^"]*" timestamp="[^"]*" classname="\([^"]*\)".*$/\2 '\3: \1'/
t end
}
d
: end

View File

@ -0,0 +1,44 @@
#! /bin/sh
#
# parse the gtest results file this replaces a sed script which produced
# the identical output. This new script is now independent of new unknown
# labels being introduced in future revisions of gtests.
#this function extracts the appropriate value from
# <testcase label="value1" label2="value2" label3="value3" />
# which value is selected from the label , which is specified
# as the 2nd parameter. The line to parse is the first parameter.
getvalue()
{
pattern1='*'${2}'="'
pattern2='"*'
front=${1#${pattern1}}
if [[ "${front}" != "${1}" ]]; then
val=${front%%${pattern2}}
# as we output the result, restore any quotes that may have
# been in the original test names.
echo ${val//&quot;/\"}
fi
}
parse()
{
while read line
do
if [[ "${line}" =~ "<testcase " ]]; then
name=$(getvalue "${line}" "name")
value=$(getvalue "${line}" "value_param")
stat=$(getvalue "${line}" "status")
class=$(getvalue "${line}" "classname")
echo "${stat} '${class}: $(echo ${name} ${value})'"
fi
done
}
# if no arguments, just take standard in, if arguments, take the args as
# files and cat them together to parse
if [ $# -eq 0 ]; then
parse
else
cat "$@" | parse
fi

View File

@ -8,7 +8,7 @@
#
# mozilla/security/nss/tests/dbtest/dbtest.sh
#
# Certificate generating and handeling for NSS QA, can be included
# Certificate generating and handeling for NSS QA, can be included
# multiple times from all.sh and the individual scripts
#
# needs to work on all Unix and Windows platforms
@ -50,6 +50,7 @@ dbtest_init()
RONLY_DIR=${HOSTDIR}/ronlydir
EMPTY_DIR=${HOSTDIR}/emptydir
CONFLICT_DIR=${HOSTDIR}/conflictdir
THREAD_DIR=${HOSTDIR}/threadir
html_head "CERT and Key DB Tests"
@ -61,7 +62,7 @@ dbtest_init()
########################################################################
dbtest_cleanup()
{
html "</TABLE><BR>"
html "</TABLE><BR>"
cd ${QADIR}
chmod a+rw $RONLY_DIR
. common/cleanup.sh
@ -78,21 +79,21 @@ dbtest_main()
{
cd ${HOSTDIR}
Echo "test opening the database read/write in a nonexisting directory"
${BINDIR}/certutil -L -X -d ./non_existent_dir
ret=$?
if [ $ret -ne 255 ]; then
html_failed "Certutil succeeded in a nonexisting directory $ret"
else
html_passed "Certutil didn't work in a nonexisting dir $ret"
html_passed "Certutil didn't work in a nonexisting dir $ret"
fi
${BINDIR}/dbtest -r -d ./non_existent_dir
ret=$?
if [ $ret -ne 46 ]; then
html_failed "Dbtest readonly succeeded in a nonexisting directory $ret"
else
html_passed "Dbtest readonly didn't work in a nonexisting dir $ret"
html_passed "Dbtest readonly didn't work in a nonexisting dir $ret"
fi
Echo "test force opening the database in a nonexisting directory"
@ -106,7 +107,7 @@ dbtest_main()
Echo "test opening the database readonly in an empty directory"
mkdir $EMPTY_DIR
${BINDIR}/tstclnt -h ${HOST} -d $EMPTY_DIR
${BINDIR}/tstclnt -h ${HOST} -d $EMPTY_DIR
ret=$?
if [ $ret -ne 1 ]; then
html_failed "Tstclnt succeded in an empty directory $ret"
@ -118,7 +119,7 @@ dbtest_main()
if [ $ret -ne 46 ]; then
html_failed "Dbtest readonly succeeded in an empty directory $ret"
else
html_passed "Dbtest readonly didn't work in an empty dir $ret"
html_passed "Dbtest readonly didn't work in an empty dir $ret"
fi
rm -rf $EMPTY_DIR/* 2>/dev/null
${BINDIR}/dbtest -i -d $EMPTY_DIR
@ -126,7 +127,7 @@ dbtest_main()
if [ $ret -ne 0 ]; then
html_failed "Dbtest logout after empty DB Init loses key $ret"
else
html_passed "Dbtest logout after empty DB Init has key"
html_passed "Dbtest logout after empty DB Init has key"
fi
rm -rf $EMPTY_DIR/* 2>/dev/null
${BINDIR}/dbtest -i -p pass -d $EMPTY_DIR
@ -134,12 +135,12 @@ dbtest_main()
if [ $ret -ne 0 ]; then
html_failed "Dbtest password DB Init loses needlogin state $ret"
else
html_passed "Dbtest password DB Init maintains needlogin state"
html_passed "Dbtest password DB Init maintains needlogin state"
fi
rm -rf $EMPTY_DIR/* 2>/dev/null
${BINDIR}/certutil -D -n xxxx -d $EMPTY_DIR #created DB
ret=$?
if [ $ret -ne 255 ]; then
if [ $ret -ne 255 ]; then
html_failed "Certutil succeeded in deleting a cert in an empty directory $ret"
else
html_passed "Certutil didn't work in an empty dir $ret"
@ -176,7 +177,7 @@ dbtest_main()
if [ $ret -ne 46 ]; then
html_failed "Dbtest r/w succeeded in a readonly directory $ret"
else
html_passed "Dbtest r/w didn't work in an readonly dir $ret"
html_passed "Dbtest r/w didn't work in an readonly dir $ret"
fi
else
html_passed "Skipping Dbtest r/w in a readonly dir because user is root"
@ -190,9 +191,9 @@ dbtest_main()
html_passed "Certutil didn't work in an readonly dir $ret"
fi
else
html_passed "Skipping Certutil delete cert in a readonly directory test because user is root"
html_passed "Skipping Certutil delete cert in a readonly directory test because user is root"
fi
Echo "test opening the database ronly in a readonly directory"
${BINDIR}/dbtest -d $RONLY_DIR -r
@ -200,7 +201,7 @@ dbtest_main()
if [ $ret -ne 0 ]; then
html_failed "Dbtest readonly failed in a readonly directory $ret"
else
html_passed "Dbtest readonly succeeded in a readonly dir $ret"
html_passed "Dbtest readonly succeeded in a readonly dir $ret"
fi
Echo "test force opening the database r/w in a readonly directory"
@ -223,7 +224,7 @@ dbtest_main()
ret=$?
if [ $ret -ne 0 ]; then
html_failed "Nicknane conflict test failed, couldn't create database $ret"
else
else
${BINDIR}/certutil -A -n alice -t ,, -i ${R_ALICEDIR}/Alice.cert -d ${CONFLICT_DIR}
ret=$?
if [ $ret -ne 0 ]; then
@ -252,7 +253,7 @@ dbtest_main()
else
html_passed "Nicknane conflict test-setting nickname conflict was correctly rejected"
fi
# import a token private key and make sure the corresponding public key is
# import a token private key and make sure the corresponding public key is
# created
${BINDIR}/pk11importtest -d ${CONFLICT_DIR} -f ${R_PWFILE}
ret=$?
@ -261,10 +262,34 @@ dbtest_main()
else
html_passed "Importing Token Private Key correctly creates the corrresponding Public Key"
fi
if [ "${NSS_DEFAULT_DB_TYPE}" = "sql" ] ; then
LOOPS=${NSS_SDB_THREAD_LOOPS-7}
THREADS=${NSS_SDB_THREAD_THREADS-30}
mkdir -p ${THREAD_DIR}
Echo "testing for thread starvation while creating keys"
${BINDIR}/certutil -N -d ${THREAD_DIR} --empty-password
${BINDIR}/sdbthreadtst -l ${LOOPS} -t ${THREADS} -d ${THREAD_DIR}
ret=$?
case "$ret" in
"0")
html_passed "Successfully completed ${LOOPS} loops in ${THREADS} threads without failure."
;;
"2")
html_failed "sdbthreadtst failed for some environment reason (like lack of memory)"
;;
"1")
html_failed "sdbthreadtst failed do to starvation using ${LOOPS} loops and ${THREADS} threads."
;;
*)
html_failed "sdbthreadtst failed with an unrecognized error code."
esac
fi
}
################## main #################################################
dbtest_init
dbtest_init
dbtest_main 2>&1
dbtest_cleanup

View File

@ -86,18 +86,8 @@ gtest_start()
fi
echo "test output dir: ${GTESTREPORT}"
echo "executing sed to parse the xml report"
sed -f "${COMMON}/parsegtestreport.sed" "$GTESTREPORT" > "$PARSED_REPORT"
echo "processing the parsed report"
cat "$PARSED_REPORT" | while read result name; do
if [ "$result" = "notrun" ]; then
echo "$name" SKIPPED
elif [ "$result" = "run" ]; then
html_passed_ignore_core "$name"
else
html_failed_ignore_core "$name"
fi
done
gtest_parse_report ${GTESTREPORT}
popd
done
}

View File

@ -133,13 +133,7 @@ ssl_gtest_start()
html_msg $? 0 "ssl_gtests ran successfully"
# Parse XML report(s).
if type xmllint &>/dev/null; then
echo "DEBUG: Using xmllint to parse GTest XML report(s)"
parse_report
else
echo "DEBUG: Falling back to legacy XML report parsing using only sed"
parse_report_legacy
fi
gtest_parse_report "${SSLGTESTREPORT}".*
}
# Helper function used when 'parallel' isn't available.
@ -148,50 +142,6 @@ parallel_fallback()
eval "${@//\{\}/0}"
}
parse_report()
{
# Check XML reports for normal test runs and failures.
local successes=$(parse_report_xpath "//testcase[@status='run'][count(*)=0]")
local failures=$(parse_report_xpath "//failure/..")
# Print all tests that succeeded.
while read result name; do
html_passed_ignore_core "$name"
done <<< "$successes"
# Print failing tests.
if [ -n "$failures" ]; then
printf "\nFAILURES:\n=========\n"
while read result name; do
html_failed_ignore_core "$name"
done <<< "$failures"
printf "\n"
fi
}
parse_report_xpath()
{
# Query the XML report with the given XPath pattern.
xmllint --xpath "$1" "${SSLGTESTREPORT}".* 2>/dev/null | \
# Insert newlines to help sed.
sed $'s/<testcase/\\\n<testcase/g' | \
# Use sed to parse the report.
sed -f "${COMMON}/parsegtestreport.sed"
}
# This legacy report parser can't actually detect failures. It always relied
# on the binary's exit code. Print the tests we ran to keep the old behavior.
parse_report_legacy()
{
while read result name && [ -n "$name" ]; do
if [ "$result" = "run" ]; then
html_passed_ignore_core "$name"
fi
done <<< "$(sed -f "${COMMON}/parsegtestreport.sed" "${SSLGTESTREPORT}".*)"
}
ssl_gtest_cleanup()
{
cd ${QADIR}