Fixes: signed integer overflow: 14 + 2147483647 cannot be represented in type 'int'
Fixes: 14794/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5677380695228416
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 10300/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-6292205497483264
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int'
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
When ff_h264_decode_mb_cavlc() failed due to wrong sl->qscale values,
e.g. dquant out of range, set the qscale to be a valid value before
returning -1 and exiting the function. The qscale value can be used
later e.g. in loop filter.
BUG=806122
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
ff_h264_decode_mb_cabac() and ff_h264_decode_mb_cavlc() are very long
functions. Declaring decode_chroma as const makes it clear the variable
doesn't change after initialization.
Signed-off-by: Wan-Teh Chang <wtc@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '3176217c60ca7828712985092d9102d331ea4f3d':
h264: decouple h264_ps from the h264 decoder
Main changes:
- a local GetBitContext is created for the various
ff_h264_decode_seq_parameter_set() attempts
- just like the old code, remove_sps() is adjusted so it doesn't remove
the pps.
Fixes decode with Ticket #631http://ffmpeg.org/pipermail/ffmpeg-user/attachments/20111108/dae58f17/attachment.mp4
but see next point as well.
- ff_h264_update_thread_context() is updated to work even when SPS
isn't set as it breaks current skip_frame code. This makes sure we
can still decode the sample from ticket #631 without the need for
-flags2 +chunks. (Thanks to Michael)
- keep {sps,pps}_ref pointers that stay alive even when the active
pps/sps get removed from the available lists (patch by michaelni with
additionnal frees in ff_h264_free_context() from mateo)
- added a check on sps in avpriv_h264_has_num_reorder_frames() to fix
crashes with mpegts_with_dvbsubs.ts from Ticket #4074http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4074/mpegts_with_dvbsubs.ts
- in h264_parser.c:h264_parse(), after the ff_h264_decode_extradata() is
called, the pps and sps from the local parser context are updated with
the pps and sps from the used h264context. This fixes fate-flv-demux.
- in h264_slice.c, "PPS changed between slices" error is not triggered
anymore in one condition as it makes fate-h264-xavc-4389 fails with
THREADS=N (Thanks to Michael)
Merged-by: Clément Bœsch <clement@stupeflix.com>
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
Merged-by: Matthieu Bouron <matthieu.bouron@stupeflix.com>
* commit 'a7da517f6a5c472f46f67dd33bb6b95ccc919923':
h264data: Move all data tables from a header to a .c file
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Make the SPS/PPS parsing independent of the H264Context, to allow
decoupling the parser from the decoder. The change is modelled after the
one done earlier for HEVC.
Move the dequant buffers to the PPS to avoid complex checks whether they
changed and an expensive copy for frame threads.
* commit 'f42485dbce614b3f63182845da43db690b427b7c':
h264: use a separate GetBitContext for slice data
Conflicts:
libavcodec/h264.c
libavcodec/h264_cavlc.c
libavcodec/h264_parser.c
libavcodec/h264_slice.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '404a416d4b1fcbf9db5569481d8181f296c01ea9':
h264: remove some remnants of data partitioning
Conflicts:
libavcodec/h264.c
libavcodec/h264_slice.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'e7226984ac13aacb84eae77a372df8ff7685848f':
h264: move [{top,left}_]cbp into the per-slice context
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '56febc993b928ccc039a32158ca60b234c311875':
h264: move the slice type variables into the per-slice context
Conflicts:
libavcodec/h264.c
libavcodec/h264_cabac.c
libavcodec/h264_cavlc.c
libavcodec/h264_slice.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
