220 Commits

Author SHA1 Message Date
Dominique Martinet
3b0668ee52 xxh*sum: print correct default algo for symlinked helpers
Printing xxh128sum's help for example would say default algo is xxh64,
which is not correct. Adjust g_defaultAlgo accordingly as well as the
currently selected algorithm.
2020-06-28 07:21:01 +02:00
Yann Collet
999e5c363e fixed and restored self-tests 2020-06-24 16:06:06 -07:00
Yann Collet
3cc3e49014 removed extraneous tests
necessarily wrong, due to ongoing changes to default secret
2020-06-23 15:00:34 -07:00
Yann Collet
ef21d43e34
Merge pull request #403 from Cyan4973/fullBlocks
small speed boost when len is multiple of block size
2020-06-22 22:00:17 -07:00
Yann Collet
3a5637f8c1 Merge branch 'newOutput074' into pi 2020-06-22 21:59:39 -07:00
Yann Collet
6e5d5d7668 do not scramble when input len is multiple of block size
also :
always use "last secret" for last stripe

changes suggested by @gzm55
2020-06-22 21:40:07 -07:00
Yann Collet
469274ab67 replaced the default kSecret by decimals of Pi
should be random enough
2020-06-22 15:48:50 -07:00
Yann Collet
adf47e6a35 Merge branch 'dev' into demerphq_xxh3 2020-06-22 10:28:11 -07:00
Yann Collet
8256aef21b
Merge pull request #399 from Cyan4973/direct_kSecret
fixed performance issue reported in #398
2020-06-22 09:57:28 -07:00
Yann Collet
71566dfcee fixed performance issue reported in #398
also : fixed XXH128_withSecret(),
which was messed up with default `kSecret`.
This error remained unnoticed because their was no test for this variant.
Added a test in `xxhsum` checking results of `XXH128_withSecret()`
2020-06-19 15:59:23 -07:00
Yann Collet
9836059130 blindfix for windows compilation
`const`ify parameters
2020-06-18 17:17:09 -07:00
Yann Collet
00833b2631 fixed incorrect assert 2020-06-18 16:13:13 -07:00
Yann Collet
a7a617584c introduced --tag for BSD-style output
hidden option for the time being (undocumented)
as it's only an output format,
there is no ability to `-c` check it yet.

The effort was however useful in refactoring `xxhsum` code base.

One advantage of the `--tag` BSD format is
that it's a lot safer to introduce additional formats.
For example, little and big-endian variants can co-exist safely,
since endianess is clearly expressed as part of the algorithm name.
It would also make it possible to introduce XXH3 (later),
even though it's 64-bit, which would be confusing with XXH64.

Also :
added ability to select algorithm by specifying bit width.
Hence :
H0 == H32 ==> XXH32
H1 == H64 ==> XXH64
H2 == H128==> XXH128
2020-06-18 16:00:10 -07:00
Yann Collet
1320ac8a4e minor refactor : simplify hash printing function
using function pointer.
2020-06-18 14:24:27 -07:00
Yann Collet
2e88626a2c minor refactor
check scenario with input size == exact nb of blocks
though not a multiple of 256 (streaming internal buffer size)
2020-06-18 12:39:53 -07:00
Yann Collet
6e1ba913bb fixed XXH128 seed avalanche for inputs of len [0-3] 2020-06-18 00:17:03 -07:00
Yann Collet
bb094a09ad fixed demerphq avalanche tests for len [0-3]
consequence : changes result of XXH3() for len [0-3]
2020-06-17 16:12:48 -07:00
Yann Collet
7259193bde
Merge pull request #392 from Cyan4973/sfvCompat
compatibility with turbosfv
2020-06-17 08:58:04 -07:00
Yann Collet
7716de4154 updated welcome string in dispatch mode
will now display "autoVec"
instead of baseline vector instruction set
2020-06-15 17:09:39 -07:00
Yann Collet
fc52938c5f compatibility with turbosfv
as requested in
https://github.com/Cyan4973/xxHash/issues/382#issuecomment-641978062
2020-06-15 10:16:39 -07:00
Yann Collet
dd79edee44 added XXH3_stream and XXH128_stream targets 2020-06-12 18:35:20 -07:00
Yann Collet
de0fe16fc7 disable staticAnalyze
it makes no sense for this test to fail now
on a completely unrelated part of the code base.
2020-06-12 18:11:01 -07:00
Yann Collet
e6683d935f fixed a completely unrelated malloc potential error 2020-06-12 16:19:51 -07:00
Yann Collet
c5867346ee bench: can select multiple variants
using comma separated IDs
2020-06-12 15:07:20 -07:00
Yann Collet
57377539bf can select multiple variants to benchmark
For example, `xxhsum -b1 -b2` will benchmark XXH32 in both aligned and unaligned mode.

Also : modified default list,
now only test main variants (XXH32, XXH64, XXH3 and XXH128).

Also : -bi0 makes it possible to generate a very quick (though inaccurate) evaluation.
2020-06-12 14:52:46 -07:00
Yann Collet
167e0c87b5 -bi0 makes a quick evaluation
modified make check to take advantage of -bi0 (faster test)
2020-06-12 12:25:15 -07:00
Yann Collet
f05d7987b6 print benchID 2020-06-12 11:39:51 -07:00
Yann Collet
37d484267a name instruction set in travisCI tests 2020-06-11 21:29:15 -07:00
Yann Collet
972ae45bf5 Merge branch 'dev' into x86dispatch_table 2020-06-11 17:52:14 -07:00
Yann Collet
16f728f058 first dispatcher concept
replace 2 functions : XXH3_64bits() and XXH3_64bits_withSeed()
new target : make dispatch
compiled with -mavx2 by default

compiles fine with -mavx512f, but then crash at runtime detection
2020-06-08 23:15:08 -07:00
Yann Collet
1b14f648d6 added auto-test for secret generator 2020-06-08 13:14:44 -07:00
Yann Collet
1d2811fc17
Merge pull request #381 from WayneD/dev
Make xxhsum check if fread() failed
2020-05-29 13:50:43 -07:00
Wayne Davison
082399bfe8 Make xxhsum check if fread() failed
If fread() fails reading a file, output an error and exit.  The new code
also avoids an update call on a 0-length buffer at end of file.
2020-05-29 12:59:27 -07:00
Yann Collet
226d3ffd88 fixed constant 2020-05-29 11:30:52 -07:00
Yann Collet
753119ebe0 generalized random ingestion pattern test 2020-05-29 10:49:51 -07:00
Yann Collet
d2a00cdabb fixed variable shadowing 2020-05-28 16:30:26 -07:00
Yann Collet
ff07228149 fix streaming checksum
Streaming implementation of XXH3/XXH128 could fail, reporting an incorrect hash value,
for certain specific ranges of length, and providing input in segments of random length.
issue reported by @WayneD
fix #378
updated test case to catch this issue
2020-05-28 16:10:00 -07:00
Witold Baryluk
43deef25ce
Remove "Loading..." from source (#373)
* Remove --quiet and "Loading ..." from manpage

* Remove "Loading ..." from markdown manpage

* Removing "Loading..." from code

* Update tests for "Loading..."

* Remove the remaining "Loading..." from source

Forgot about this one

* Test error message on non-existing file

* Fix a typo in a test (xxhsum vs ./xxhsum)

* Fix a typo for real. noneexistent -> nonexistent

Tested.
2020-05-27 11:43:41 -07:00
Yann Collet
29ff35da80 Improved help message
following suggestions from #345.

Note : only cosmetic changes in this diff,
I've avoided any change that would impact the behavior of the program.
2020-04-07 15:19:08 -07:00
Yann Collet
fa1368ad7a
Merge pull request #346 from easyaspi314/unicode_fix_3
Fix Unicode hopefully for the last time
2020-04-07 14:18:10 -07:00
James Z.M. Gao
4b12376862 add optimized implementation for AVX512 targets 2020-04-06 15:58:26 +08:00
easyaspi314 (Devin)
ab2862f867 Fix Unicode hopefully for the last time
Use `__wgetmainargs()` instead of `GetCommandLineW()` and
`CommandLineToArgvW()`. While it is an internal API, aside from
declaring the function, it actually simplifies things a lot, as we can
setup a `wmain()` equivalent in a few lines.

It also avoids linking to `Shell32.dll`, which is a lot of bloat for a
simple command line utility.

Most importantly, this supports wildcards on `cmd.exe`, fixing #341.

Also improved comments and did minor cleanup.
2020-03-28 15:40:51 -04:00
easyaspi314 (Devin)
7ea2940fd6 Fix a few warnings
- Trailing comma in enum in xxhsum.c
 - Extra semicolon
 - Non-constant brace list initializers
2020-03-16 12:29:49 -04:00
easyaspi314 (Devin)
7328e217af Re-wrap comments.
Helps when you set your guideline to the correct column.
2020-03-10 14:12:15 -04:00
easyaspi314 (Devin)
a532573cf9 New Windows Unicode solution, works on XP
Now uses WriteConsoleW instead of vfwprintf. This is codepage
independent and works on Windows XP.

Doing this also avoids the _O_U8TEXT hack.
2020-03-10 14:06:09 -04:00
easyaspi314 (Devin)
962813dc7d Use better wording for short hash name
XXH3_64b seeded -> XXH3_64b w/seed
XXH3_64b secret -> XXH3_64b w/secret
2020-03-09 21:26:25 -04:00
easyaspi314 (Devin)
e30c5cc908 xxhsum.c: Remove some unused macros
MEM_MODULE and SET_SPARSE_FILE_MODE aren't even used, remove them.
2020-03-09 20:03:15 -04:00
easyaspi314 (Devin)
3e567ea317 Add withSecret variants to the benchmark
Moved sanityBuffer generator to its own function and made a static
buffer to use as a secret. Easier than passing a pointer around.
2020-03-09 19:44:04 -04:00
easyaspi314 (Devin)
a5e21d1da6 Abstract xxhsum bench, increase line length
Uses a table and a loop to reduce copy/paste and allow easy testing of
other hash functions. Create a wrapper and insert it into
g_hashesToBench, and it will automatically be added to the benchmark.

The hash display line has been made longer to actually fit xxHash's
names instead of clipping them. It is also configurable.
2020-03-09 19:02:48 -04:00
Yann Collet
1dc959b74d
Merge pull request #332 from easyaspi314/documentation-v2
More documentation and cleanup
2020-03-09 15:16:40 -07:00