Commit Graph

141 Commits

Author SHA1 Message Date
Lynne 3655d8bcc6 diracdec: rewrite golomb reader
This version is able to output multiple coefficients at a time and
is able to altogether remove actual golomb code parsing.
Its also able to partially recover the last coefficient in case
the packet is incomplete.

Total decoder performance gain for 8bit 420 1080p lossless: 40%.
Total decoder performance gain for 10bit 420 1080p lossless: 40%.

clang was able to vectorize the loop much better than
my handwritten assembly, but gcc was very naive and didn't.

Lookup table is a rewritten version of vc2hqdecode.
2020-03-12 20:26:48 +00:00
Michael Niedermayer 850782f9fa avcodec/diracdec: Check that slices are fewer than pixels
Fixes: Timeout (197sec ->144ms)
Fixes: 15034/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5733549405110272

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-13 22:14:03 +02:00
Michael Niedermayer e2bd17a4a8 avcodec/diracdec: Fix integer overflow in global_mv()
Fixes: signed integer overflow: 16384 * 196607 cannot be represented in type 'int'
Fixes: 14810/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5091232683917312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-06-05 13:19:59 +02:00
Michael Niedermayer 1680387c57 avcodec/diracdec: Check for arith decoder errors in dirac_unpack_block_motion_data()
Fixes: Timeout (54sec -> 188ms)
Fixes: 14585/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5649933052411904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-05-27 19:01:38 +02:00
Michael Niedermayer a1acaeed57 avcodec/diracdec: Use 64bit in intermediate of global motion vector field generation
It seems the specification does not limit the value to 32bit

Fixes: signed integer overflow: -109611143 * 24 cannot be represented in type 'int'
Fixes: 13477/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5648337460527104

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-04-19 21:06:09 +02:00
Michael Niedermayer 3477bbb041 avcodec/diracdec: Count truncated parts as errors in decode_component()
Fixes: Timeout (29sec -> 4sec)
Fixes: 13150/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5690185671507968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-03-17 11:35:00 +01:00
Michael Niedermayer c0b62acbcc avcodec/diracdec: Correct max pixels check
Dirac internally allocates 5 images per plane and frame currently. One being the actual
image the other 4 being filtered for motion compensation.

Fixes: Out of memory
Fixes: 12870/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5684825871089664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-03-03 13:17:02 +01:00
Michael Niedermayer ab9aa27d45 avcodec/diracdec: Propagate errors from dirac_get_arith_uint()
Testcase: 11663/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5636791864918016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-01-16 01:29:18 +01:00
Michael Niedermayer 2f483299fe avcodec/diracdec: Check component quant
Fixes: Timeout
Fixes: 10708/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5730140957442048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-11-27 14:38:03 +01:00
Michael Niedermayer 990a4d4507 avcodec/diracdec: check that GetBitContext has not ended in codeblock()
Fixes: Timeout (part 2 of 2)
Fixes: 9774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5748957085958144

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-09-28 01:35:12 +02:00
Michael Niedermayer 7bf3e2b743 avcodec/diracdec: Propagate errors from codeblock()
Consider a component to be damaged if more than 50% of its subbands are damaged

Fixes: Timeout (part 1 of 2)
Fixes: 9774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5748957085958144

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-09-28 01:35:09 +02:00
Michael Niedermayer 134932ef62 avcodec/diracdec: Check bytes count in else branch in decode_lowdelay() too
Fixes: signed integer overflow: 8 * 340018243 cannot be represented in type 'int'
Fixes: 9441/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5194665207791616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-28 14:35:51 +02:00
Michael Niedermayer 41d1983276 avcodec/diracdec: Check slice numbers for overflows in relation to picture dimensions
Fixes: signed integer overflow: 88 * 33685506 cannot be represented in type 'int'
Fixes: 9433/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5725943535501312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-28 14:35:50 +02:00
Michael Niedermayer bc02937689 avcodec/diracdec: Change frame_number to 64bit as its a 32bit from the bitstream and we also have a -1 special case
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 9291/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6324345860259840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-28 14:35:50 +02:00
Michael Niedermayer 7a50a469c4 avcodec/diracdec: Prevent integer overflow in intermediate in global_mv()
Fixes: signed integer overflow: -393471 * 5460 cannot be represented in type 'int'
Fixes: 8890/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6299775379963904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-28 14:35:50 +02:00
Michael Niedermayer b8c536c13f avcodec/diracdec: Fix integer overflow in mv computation
Fixes: signed integer overflow: -2072 + -2147483646 cannot be represented in type 'int'
Fixes: 6097/clusterfuzz-testcase-minimized-5034145253163008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-02-20 15:27:51 +01:00
Michael Niedermayer d222625884 avcodec/diracdec: Use int64 in global mv to prevent overflow
Fixes: runtime error: signed integer overflow: 361 * -6295541 cannot be represented in type 'int'
Fixes: 5911/clusterfuzz-testcase-minimized-6450382197751808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-02-20 15:27:51 +01:00
Michael Niedermayer 45a1b71f7b avcodec/diracdec: Fix integer overflow with quant
Fixes: signed integer overflow: 2 + 2147483646 cannot be represented in type 'int'
Fixes: 4792/clusterfuzz-testcase-minimized-6322450775146496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-01-08 21:50:12 +01:00
Nicolas Frattaroli 0224824147 diracdec: fix deprecated API usage
avcodec_get_chroma_sub_sample is deprecated and generates a warning
during build, so av_pix_fmt_get_chroma_sub_sample is used

Signed-off-by: Nicolas Frattaroli <ffmpeg@fratti.ch>
Signed-off-by: James Almer <jamrial@gmail.com>
2017-11-03 00:26:26 -03:00
Michael Niedermayer c9f7992494 avcodec/diracdec: Fix overflow in DC computation
Fixes: runtime error: signed integer overflow: 11896 + 2147483646 cannot be represented in type 'int'
Fixes: 3053/clusterfuzz-testcase-minimized-6355082062856192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-09-11 12:23:53 +02:00
Michael Niedermayer f779435db7 avcodec/diracdec: Fix integer overflow in INTRA_DC_PRED()
Fixes: runtime error: signed integer overflow: 1168175789 + 1168178473 cannot be represented in type 'int'
Fixes: 3081/clusterfuzz-testcase-minimized-4807564879462400
Fixes: 2844/clusterfuzz-testcase-minimized-5561715838156800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-09-02 23:50:28 +02:00
Michael Niedermayer ff5774be9a avcodec/diracdec: Fixes integer overflow
Fixes: runtime error: signed integer overflow: 340018243 * 27 cannot be represented in type 'int'
Fixes: 2861/clusterfuzz-testcase-minimized-5361070510178304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-17 00:23:01 +02:00
Michael Niedermayer dd8cd8dae9 avcodec/diracdec: Check perspective_exp and zrs_exp.
Fixes: undefined shift
Fixes: runtime error: shift exponent 264 is too large for 32-bit type 'int'
Fixes: 2860/clusterfuzz-testcase-minimized-4672811689836544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-17 00:23:01 +02:00
Michael Niedermayer a69061fc98 avcodec/diracdec: Check weight_log2denom
Fixes: runtime error: shift exponent -1 is negative
Fixes: 2742/clusterfuzz-testcase-minimized-5724322402402304
Fixes: 2744/clusterfuzz-testcase-minimized-4672435653705728
Fixes: 2749/clusterfuzz-testcase-minimized-5298741273690112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-01 13:54:43 +02:00
Michael Niedermayer 2fc3a9d16a avcodec/diracdec: Fix integer overflow in divide3()
Fixes: runtime error: signed integer overflow: -1073746548 * 21845 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-07-29 04:05:53 +02:00
Michael Niedermayer aedb1a6d3d avcodec/diracdec: Fix integer overflow in signed multiplication in UNPACK_ARITH()
Fixes: runtime error: signed integer overflow: 1073741823 * 4 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-07-28 03:35:22 +02:00
Michael Niedermayer b67804510a avcodec/diracdec: Check dimensions which are closer to what is allocated in alloc_sequence_buffers()
Fixes OOM
Fixes: 2674/clusterfuzz-testcase-minimized-4999700518273024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-07-21 17:37:05 +02:00
Michael Niedermayer d8b35fef08 avcodec/diracdec: Fix off by 1 error in quant check
Fixes: out of array read
Fixes: 1781/clusterfuzz-testcase-minimized-4617176877105152

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-25 19:32:39 +02:00
Michael Niedermayer bd3e6314e0 avcodec/diracdec: Factor quant matrix reads
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-25 19:32:39 +02:00
Michael Niedermayer 48ca25d02b avcodec/diracdec: Fix Assertion frame->buf[0] failed at libavcodec/decode.c:610
Fixes: 1487/clusterfuzz-testcase-minimized-6288036495097856

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-11 23:55:59 +02:00
Ronald S. Bultje 6146c7d8f6 dirac: make initialization of arithmetic coder tables threadsafe. 2017-03-28 11:21:27 -04:00
Andreas Cadhalpun 1071528346 diracdec: check return code of get_buffer_with_edge
If it fails, buffers aren't allocated, causing NULL pointer dereferencing.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-04 20:35:23 +01:00
Andreas Cadhalpun 02a3660717 diracdec: clear slice_params_num_buf on allocation failure
Otherwise it can be non-zero next time decode_lowdelay is called, causing
slice_params_buf not to be allocated, leading to a NULL pointer dereference.

The problem was introduced in commit
b343023f68.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-04 20:34:51 +01:00
Andreas Cadhalpun 457db0c6a8 diracdec: use correct buffer for slice_params_buf realloc
This fixes a double-free detected by AddressSanitizer.

The problem was introduced in commit
b343023f68.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-04 20:34:38 +01:00
Michael Niedermayer 3f04b9d398 avcodec/diracdec: Check numx/y
Fixes division by 0
Fixes: 60261c4469ba3e11059890fb2832a515/asan_generic_135e694_2790_beb94eaa0aeb7d11c0437375a8964a99.drc

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-20 19:36:01 +02:00
Rostislav Pehlivanov 65a1e55ffd diracdec: fix unchecked byte length
Also drops the start variable since it's redundant.
Found by Coverity, fixes CID1363964

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2016-07-13 23:53:05 +01:00
Rostislav Pehlivanov 46030b3f1a diracdec: fix maximum quantization index checks
Found by Coverity, fixes CID1363961 and CID1363962

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2016-07-13 23:40:06 +01:00
Rostislav Pehlivanov 4773c9f6a5 diracdec: fix #coeffs -> byte conversion
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2016-07-12 10:44:24 +01:00
Rostislav Pehlivanov ea75af3ca5 diracdec: do not memset the entire coefficient buffer for HQ pictures
This is now handled by the slice decoding function.

Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:41:32 +01:00
Rostislav Pehlivanov b343023f68 diracdec: do not allocate and free slice parameters every frame
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:40:57 +01:00
Rostislav Pehlivanov 2b970bbe9b diracdec: implement a LUT-based Golomb code parser
Still much left to optimize, but it provides a significant performance
improvement - 10% for 300Mbps (1080p30), 25% for 1.5Gbps (4k 60fps) in
comparison with the default implementation.

Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:40:28 +01:00
Rostislav Pehlivanov 492ee9b685 diracdec: rewrite HQ slice decoding
Now coefficients are written to a buffer and are then dequantized by the
new SIMD dequantization functions. For the lower bands without enough
coefficients to fill a register (and hence they overwrite) the C version
of the dequantization function is used.

The buffer is per-thread and will be realloc'd if anything changes.
This prevents regressions and having to limit slice size.

Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:38:45 +01:00
Rostislav Pehlivanov 31d37be930 diractab: expose the maximum quantization index as a macro
Prevents having to have random magic values in the decoder and a
separate macro in the encoder.

Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:38:01 +01:00
Rostislav Pehlivanov 410c92ccea diracdec: decode HQ profile slices in rows
Siginificantly improves the performance.

Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:34:54 +01:00
Rostislav Pehlivanov cabd98572d diracdec: simplify golomb parsing and dequantization
In preparation for the following commits, this commit simplifies the
coefficient parsing and dequantization function. It was needlessly
inlined without much performance gain.

Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
2016-07-11 23:34:25 +01:00
Hendrik Leppkes 5b5be3e4e3 Merge commit '4024b566d664a4b161d677554be52f32e7ad4236'
* commit '4024b566d664a4b161d677554be52f32e7ad4236':
  golomb: Give svq3_get_se_golomb()/svq3_get_ue_golomb() better names

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2016-06-26 15:12:48 +02:00
Michael Niedermayer 681879c02f avcodec/diracdec: Fix potential integer overflow
Fixes CID1361948

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-05-19 18:13:29 +02:00
Michael Niedermayer 8f2a1990c0 avcodec/diracdec: check bitstream size related fields for overflows
Fixes segfault
Fixes Ticket5333

Regression since bfc8a4dabe
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-03-28 04:39:57 +02:00
Timothy Gu 671761d713 diracdec: Pass DWTPlane to dwt init 2016-02-07 09:09:13 -08:00
Timothy Gu e04912c0b6 diracdec: Split DWTPlane struct from Plane 2016-02-07 09:09:13 -08:00