Commit Graph

244 Commits

Author SHA1 Message Date
Yann Collet
0208d72ea2
Merge pull request #427 from Cyan4973/benchStreamSeed
Improves speed of streaming variant on small inputs
2020-07-17 20:08:24 -07:00
Jason Kim
fac98dab6b style: comment on breaks
Comment on the `break`s that we are skipping parsing the rest of the
arguments so that they are treated strictly as file names.
2020-07-13 23:52:28 -07:00
Jason Kim
b4c092cd75 feat: support dash dash
Support dash dash (`--`) to mark the rest of the arguments as filenames.
Note this special case (an expected limitation):

```sh
touch a
./xxhsum a --
```

```
ef46db3751d8e999  a
Error: Could not open '--': No such file or directory.
```
2020-07-13 23:48:33 -07:00
Jason Kim
ad10c2f9a0 fix: don't interpret options in file list
When parsing arguments, the first non-option is treated as the start of
a list of file names.  However, arguments continue to be parsed, so any
later arguments that look like options will have a dual meaning as an
option and file name.  This is unexpected behavior, and here are two
solutions:

- Treat later option-like arguments as options
- Treat later option-like arguments as file names

Go with the latter solution for now because it is easier.  The former
solution is preferred because it is more flexible and similar to how
`shasum` parses its arguments.
2020-07-13 21:57:39 -07:00
Yann Collet
18f0991cef do not regenerate secret from seed
when state was already used with the same `seed` previously.
This skips secret generation when `seed` remains constant.

Also : improves streaming speed on small data
thanks to lighter memset() during reset().
2020-07-10 22:11:45 -07:00
Yann Collet
289813fab9 added benchmark tests for streaming with seed 2020-07-10 16:09:02 -07:00
Yann Collet
0a7508e80b
Merge pull request #425 from devpow112/patch-1
Remove uninitialized local variable warning
2020-07-06 23:40:46 -07:00
Yann Collet
96d2649d50
Merge pull request #424 from Cyan4973/skipDir
can skip over directory
2020-07-06 23:39:37 -07:00
Devon Powell
0c3c6c9e85
Remove uninitialized local variable warning
Can generate a `potentially uninitialized local variable 'finalHash' used` warning.
2020-07-06 21:35:33 -04:00
Yann Collet
cf4ac6b6a8 can skip over directory
`xxhsum *` would choke over directory entries.
This is because such a name still `fopen()` correctly,
but then reading its content fails.
This patch detects this situation, and just skip it, with a message.
`xxhsum` will also return an error code 1.
2020-07-06 18:20:07 -07:00
Yann Collet
bf45b4d5b5
Merge pull request #423 from WayneD/allow-multi
Allow multiple hash types per file
2020-07-06 18:00:54 -07:00
Wayne Davison
63cbb8b715 Say "line is" and "lines are". 2020-07-06 14:13:41 -07:00
Wayne Davison
5d6c150401 Don't reject multiple hash types per file; add more xxhsum tests. 2020-07-06 14:10:52 -07:00
Wayne Davison
62c7912c26 Fix & improve some comments
Also ensure the ParsedLine values are reset before returning an error.
2020-07-06 13:38:50 -07:00
Yann Collet
f2b41dd51e
Merge pull request #420 from WayneD/dev
Add the ability to parse BSD style --check lines.
2020-07-06 12:41:49 -07:00
Devon Powell
bca0ce8a2f
Remove pre-processor already defined warning
When including into other source code (especially when using cmake or the like)
the declaration of _CRT_SECURE_NO_WARNINGS in `xxhsum.c` can cause a compiler
warning about already defined pre-processor symbols. Wrap the symbol definition
in a check to see if it's already defined and not try to define it again.
2020-07-05 23:29:50 -04:00
Wayne Davison
7032ced649 Change reverseDigits -> reverseBytes. 2020-07-05 19:25:15 -07:00
Wayne Davison
cfed37c9c3 Also allow --check --little-endian for non-tag. 2020-07-05 18:27:50 -07:00
Wayne Davison
eb13bcf1c9 Handle the --check of a --tag --little-endian output. 2020-07-05 18:18:59 -07:00
Wayne Davison
e37535169e Cast pointer difference to size_t. 2020-07-05 17:09:32 -07:00
Wayne Davison
d6222723bd Declare hash_len as a size_t. 2020-07-05 16:50:34 -07:00
Wayne Davison
213f72ecd3 Reject a too-short line that ends after the first space.
The prior code accepted a --check line that ends after the first space
and returned the filename from the prior line.  The new code rejects the
line as invalid.
2020-07-05 16:46:21 -07:00
Wayne Davison
5ffd33ef3a Add the ability to parse BSD style --check lines. 2020-07-05 15:49:48 -07:00
Yann Collet
be54f1f793 minor welcome message tweaks
- display binary name without path
- only provide binary name and version by default
- print extended compilation information on benchmark and --version
2020-07-02 17:02:38 -07:00
Dominique Martinet
3b0668ee52 xxh*sum: print correct default algo for symlinked helpers
Printing xxh128sum's help for example would say default algo is xxh64,
which is not correct. Adjust g_defaultAlgo accordingly as well as the
currently selected algorithm.
2020-06-28 07:21:01 +02:00
Yann Collet
999e5c363e fixed and restored self-tests 2020-06-24 16:06:06 -07:00
Yann Collet
3cc3e49014 removed extraneous tests
necessarily wrong, due to ongoing changes to default secret
2020-06-23 15:00:34 -07:00
Yann Collet
ef21d43e34
Merge pull request #403 from Cyan4973/fullBlocks
small speed boost when len is multiple of block size
2020-06-22 22:00:17 -07:00
Yann Collet
3a5637f8c1 Merge branch 'newOutput074' into pi 2020-06-22 21:59:39 -07:00
Yann Collet
6e5d5d7668 do not scramble when input len is multiple of block size
also :
always use "last secret" for last stripe

changes suggested by @gzm55
2020-06-22 21:40:07 -07:00
Yann Collet
469274ab67 replaced the default kSecret by decimals of Pi
should be random enough
2020-06-22 15:48:50 -07:00
Yann Collet
adf47e6a35 Merge branch 'dev' into demerphq_xxh3 2020-06-22 10:28:11 -07:00
Yann Collet
8256aef21b
Merge pull request #399 from Cyan4973/direct_kSecret
fixed performance issue reported in #398
2020-06-22 09:57:28 -07:00
Yann Collet
71566dfcee fixed performance issue reported in #398
also : fixed XXH128_withSecret(),
which was messed up with default `kSecret`.
This error remained unnoticed because their was no test for this variant.
Added a test in `xxhsum` checking results of `XXH128_withSecret()`
2020-06-19 15:59:23 -07:00
Yann Collet
9836059130 blindfix for windows compilation
`const`ify parameters
2020-06-18 17:17:09 -07:00
Yann Collet
00833b2631 fixed incorrect assert 2020-06-18 16:13:13 -07:00
Yann Collet
a7a617584c introduced --tag for BSD-style output
hidden option for the time being (undocumented)
as it's only an output format,
there is no ability to `-c` check it yet.

The effort was however useful in refactoring `xxhsum` code base.

One advantage of the `--tag` BSD format is
that it's a lot safer to introduce additional formats.
For example, little and big-endian variants can co-exist safely,
since endianess is clearly expressed as part of the algorithm name.
It would also make it possible to introduce XXH3 (later),
even though it's 64-bit, which would be confusing with XXH64.

Also :
added ability to select algorithm by specifying bit width.
Hence :
H0 == H32 ==> XXH32
H1 == H64 ==> XXH64
H2 == H128==> XXH128
2020-06-18 16:00:10 -07:00
Yann Collet
1320ac8a4e minor refactor : simplify hash printing function
using function pointer.
2020-06-18 14:24:27 -07:00
Yann Collet
2e88626a2c minor refactor
check scenario with input size == exact nb of blocks
though not a multiple of 256 (streaming internal buffer size)
2020-06-18 12:39:53 -07:00
Yann Collet
6e1ba913bb fixed XXH128 seed avalanche for inputs of len [0-3] 2020-06-18 00:17:03 -07:00
Yann Collet
bb094a09ad fixed demerphq avalanche tests for len [0-3]
consequence : changes result of XXH3() for len [0-3]
2020-06-17 16:12:48 -07:00
Yann Collet
7259193bde
Merge pull request #392 from Cyan4973/sfvCompat
compatibility with turbosfv
2020-06-17 08:58:04 -07:00
Yann Collet
7716de4154 updated welcome string in dispatch mode
will now display "autoVec"
instead of baseline vector instruction set
2020-06-15 17:09:39 -07:00
Yann Collet
fc52938c5f compatibility with turbosfv
as requested in
https://github.com/Cyan4973/xxHash/issues/382#issuecomment-641978062
2020-06-15 10:16:39 -07:00
Yann Collet
dd79edee44 added XXH3_stream and XXH128_stream targets 2020-06-12 18:35:20 -07:00
Yann Collet
de0fe16fc7 disable staticAnalyze
it makes no sense for this test to fail now
on a completely unrelated part of the code base.
2020-06-12 18:11:01 -07:00
Yann Collet
e6683d935f fixed a completely unrelated malloc potential error 2020-06-12 16:19:51 -07:00
Yann Collet
c5867346ee bench: can select multiple variants
using comma separated IDs
2020-06-12 15:07:20 -07:00
Yann Collet
57377539bf can select multiple variants to benchmark
For example, `xxhsum -b1 -b2` will benchmark XXH32 in both aligned and unaligned mode.

Also : modified default list,
now only test main variants (XXH32, XXH64, XXH3 and XXH128).

Also : -bi0 makes it possible to generate a very quick (though inaccurate) evaluation.
2020-06-12 14:52:46 -07:00
Yann Collet
167e0c87b5 -bi0 makes a quick evaluation
modified make check to take advantage of -bi0 (faster test)
2020-06-12 12:25:15 -07:00