Commit Graph

326 Commits

Author SHA1 Message Date
Yann Collet
2c992d0cc7 merge xxh3 implementation within xxhash.h
xxh3.h is no longer necessary
but remains available,
in case some programs do #include "xxh3.h" directly.

Also : bumped version number to v0.8.0
2020-07-23 10:24:51 -07:00
Yann Collet
76ab5f4d1e fix minor warning
strict C90
2020-07-18 22:32:17 -07:00
Yann Collet
662b223eef slightly improved streaming speed on small
by reducing memset() even more

note : required re-organizing members order
2020-07-10 23:34:17 -07:00
Yann Collet
186a3dfb61 fix minor c++ cast warning 2020-07-10 23:20:16 -07:00
Yann Collet
18f0991cef do not regenerate secret from seed
when state was already used with the same `seed` previously.
This skips secret generation when `seed` remains constant.

Also : improves streaming speed on small data
thanks to lighter memset() during reset().
2020-07-10 22:11:45 -07:00
Yann Collet
e4ecae117c fixed dispatcher compilation in c++ mode 2020-07-02 16:35:52 -07:00
Yann Collet
6379a93461 fix _mm512_shuffle_epi32 in c++ mode
added relevant test in CI
2020-07-02 15:50:49 -07:00
easyaspi314 (Devin)
6c82248bdb Fix C99isms in XXH3_initCustomSecret_avx512
Removed // comments and a compound literals.
2020-06-27 23:03:19 -04:00
Mattias Ellert
2677da37d5 Fix compiler warning for 32 bit architectures
In file included from xxhash.h:2065,
                 from xxhash.c:43:
xxh3.h: In function 'XXH3_len_9to16_128b':
xxh3.h:2278: warning: integer constant is too large for 'long' type
2020-06-27 20:05:58 +02:00
Yann Collet
6ab5932983 XXH3 mixer is now the same as XXH128
Simplify code
2020-06-24 13:07:31 -07:00
Yann Collet
9548436582 use 128-bit mixing even in 64-bit mode 2020-06-24 00:41:16 -07:00
Yann Collet
be32411fcc reverted default secret to Farsh 2020-06-23 14:46:48 -07:00
Yann Collet
ef21d43e34
Merge pull request #403 from Cyan4973/fullBlocks
small speed boost when len is multiple of block size
2020-06-22 22:00:17 -07:00
Yann Collet
3a5637f8c1 Merge branch 'newOutput074' into pi 2020-06-22 21:59:39 -07:00
Yann Collet
6e5d5d7668 do not scramble when input len is multiple of block size
also :
always use "last secret" for last stripe

changes suggested by @gzm55
2020-06-22 21:40:07 -07:00
Yann Collet
469274ab67 replaced the default kSecret by decimals of Pi
should be random enough
2020-06-22 15:48:50 -07:00
Yann Collet
adf47e6a35 Merge branch 'dev' into demerphq_xxh3 2020-06-22 10:28:11 -07:00
Yann Collet
71566dfcee fixed performance issue reported in #398
also : fixed XXH128_withSecret(),
which was messed up with default `kSecret`.
This error remained unnoticed because their was no test for this variant.
Added a test in `xxhsum` checking results of `XXH128_withSecret()`
2020-06-19 15:59:23 -07:00
Yann Collet
2e88626a2c minor refactor
check scenario with input size == exact nb of blocks
though not a multiple of 256 (streaming internal buffer size)
2020-06-18 12:39:53 -07:00
Yann Collet
6e1ba913bb fixed XXH128 seed avalanche for inputs of len [0-3] 2020-06-18 00:17:03 -07:00
Yann Collet
bb094a09ad fixed demerphq avalanche tests for len [0-3]
consequence : changes result of XXH3() for len [0-3]
2020-06-17 16:12:48 -07:00
Yann Collet
972ae45bf5 Merge branch 'dev' into x86dispatch_table 2020-06-11 17:52:14 -07:00
Yann Collet
125710950d fix avx512 dispatch 2020-06-11 00:01:39 -07:00
Yann Collet
c94b87eccd fixed minor warnings 2020-06-10 18:12:08 -07:00
Yann Collet
e9a9c7f753 fix minor cast warning 2020-06-10 16:51:10 -07:00
Yann Collet
f533356c96 fixed assert 2020-06-10 16:42:27 -07:00
Yann Collet
0f3ba3f7e4 added XXH128_withSeed 2020-06-10 16:30:12 -07:00
Yann Collet
958abdc9a6 added XXH128_withSecret 2020-06-10 15:58:56 -07:00
Yann Collet
b4ea4f53c7 added XXH128 2020-06-10 15:43:37 -07:00
Yann Collet
21a9a8c995 added XXH3_64bits_update_dispatch 2020-06-10 12:12:24 -07:00
Yann Collet
7c8b6995ff simplified seed allocation
to be compatible with any vector mode selected by dispatch.
2020-06-10 11:36:58 -07:00
Yann Collet
df5aca4f85 shortened source code for small inputs 2020-06-10 10:51:33 -07:00
Yann Collet
a80dac4426 added sse2 target 2020-06-09 14:41:51 -07:00
Yann Collet
0e87760b03 fix compilation strategy
starting from "baseline",
adding support for additional vector instructions selectively
2020-06-09 14:10:13 -07:00
Yann Collet
16f728f058 first dispatcher concept
replace 2 functions : XXH3_64bits() and XXH3_64bits_withSeed()
new target : make dispatch
compiled with -mavx2 by default

compiles fine with -mavx512f, but then crash at runtime detection
2020-06-08 23:15:08 -07:00
Yann Collet
726e4fa4f4 minor : improve comment and code readability
as suggested in c546d08ff3 (r39708108)

Also :
fix : `seeds[]` was incorrectly initialized.
2020-06-07 23:11:07 -07:00
Yann Collet
c546d08ff3 kdf v3
Following comments from @koraa :
- fix endianess issue, by using canonical representation
- all segments are derived from first one, in order to reduce dependency chain
- all derived segments use a seed, which is a combination raw custom seed content and segNb
- updated documentation

This variant breaks the possibility for an actor
to derive the entire secret from the knowledge of one of its segments
since it requires the seed, which is derived from original custom seed,
which is not present anywhere, condensed in the scrambling operation.
2020-06-05 16:46:55 -07:00
Yann Collet
58922a773b secret generator v2
update 128-bit scrambler at each 16-bytes round.
ensure there is no delta-correlation possible between segments,
even when the initial custom seed is poor (<= 16 bytes, or full of `\0` characters).
2020-06-05 00:37:56 -07:00
Yann Collet
4d0ec6e6ed first version of kdf 2020-06-02 22:41:24 -07:00
Yann Collet
2a80fa27a0
Merge pull request #379 from Cyan4973/fix378
fix streaming checksum
2020-05-29 13:16:09 -07:00
Yann Collet
be37c619bf added aarch64 + clang tests
fixed the const property for the asm hack to work
2020-05-28 18:01:11 -07:00
Yann Collet
ff07228149 fix streaming checksum
Streaming implementation of XXH3/XXH128 could fail, reporting an incorrect hash value,
for certain specific ranges of length, and providing input in segments of random length.
issue reported by @WayneD
fix #378
updated test case to catch this issue
2020-05-28 16:10:00 -07:00
Yann Collet
0ce6bc0f4a conflict fixes and minor code style changes
added avx512 CI tests
2020-05-27 20:35:07 -07:00
Yann Collet
0bced63140
vectorize XXH3_initCustomSecret (#376)
* vectorize initCustomSecret method

* revert Makefile

* pass compiling

* revert Makefile

* fix unused function warning

* do not use brace initialization for sse types

* workaround for msvc 32bit mode before vs 2013U2

* workaround before MSVC 2015

* use portable way for sse2

* Revert "use portable way for sse2"

This reverts commit 34eee07a7071ab963bd5a48e3960f40b28e209c2.

* use _mm_load_si128 for MSVC x86

* try to minimize align of custom secret

* for avx2 and sse2, optmize the prolog of XXH3_hashLong_xb_withSeed

Co-authored-by: James Z.M. Gao <gaozhiming@360.cn>
2020-05-27 18:44:06 -07:00
Yann Collet
ceed8a7153
simplified XXH_PREFETCH_DIST logic (#375)
still distinguishes :
- __clang__ : 320
- AVX512 : 512
- the rest : 384
2020-05-27 15:00:40 -07:00
Yann Collet
e3ad2c907f support for secretSize > 4 GB in streaming mode
fix #368
2020-05-20 19:21:25 -07:00
Yann Collet
6af67b92b7 Merge branch 'dev' into moveXXH3state 2020-05-20 14:31:11 -07:00
easyaspi314 (Devin)
9405c1ca88 Fix PPC compile errors 2020-05-20 16:49:36 -04:00
easyaspi314 (Devin)
3eb1a010a0 Clean up namespace
- Every symbol, typedef, and macro is prefixed with XXH or xxh.
 - For compatibility with legacy code that messes with the internals,
   XXH_OLD_NAMES can be defined to wrap the old names in macros. This
   also brings back U32 and friends.
2020-05-20 16:36:24 -04:00
Yann Collet
deeae7f5fd fixed minor conversion warning 2020-05-20 13:05:34 -07:00