40514 Commits

Author SHA1 Message Date
Karsten Otto
3bf39f2aef libavcodec/mpegaudiodecheader.h : detect reserved mpeg id
Check the MPEG version ID for the reserved bit pattern 01, and abort the
header check in that case. This reduces the chance of misinterpreting
arbitrary data as a valid header, and prevents resulting audio artifacts.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-09 03:25:53 +02:00
Michael Niedermayer
ee1e3ca5eb avcodec/mjpegdec: Check for odd progressive RGB
Fixes: out of array access
Fixes: 9225/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEGLS_fuzzer-5684770334834688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-08 00:55:49 +02:00
Michael Niedermayer
115b7c96e9 libavcodec/vp8: Do not compute line pointers per pixel in fade()
72->60 seconds
Testcase: 8680/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5861504418054144

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-07 12:09:34 +02:00
Michael Niedermayer
284dde24da avcodec/vp8_parser: Do not leave data/size uninitialized
This is identical to what the VP9 parser does

Fixes: 9215/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBVPX_VP8_fuzzer-5768227253649408
Fixes: out of memory access

This may also fix oss fuzz issue 9212

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-07 12:09:34 +02:00
Michael Niedermayer
5aba5b89d0 avcodec/mpeg4videodec: Check for bitstream end in read_quant_matrix_ext()
Fixes: out of array read
Fixes: asff-crash-0e53d0dc491dfdd507530b66562812fbd4c36678

Found-by: Paul Ch <paulcher@icloud.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 23:14:24 +02:00
James Almer
a61b56624b avcodec/atrac9tab: add missing header include
Fixes make checkheaders

Signed-off-by: James Almer <jamrial@gmail.com>
2018-07-04 15:56:44 -03:00
Carl Eugen Hoyos
e25c25ebd8 lavc/atrac9tab: Add inclusion guards.
Fixes fate-source.
2018-07-04 18:54:44 +02:00
Michael Niedermayer
267ba2aa96 avcodec/indeo4: Check for end of bitstream in decode_mb_info()
Fixes: Timeout
Fixes: 8776/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO4_fuzzer-5361788798369792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 12:15:10 +02:00
Michael Niedermayer
4423085ca5 avcodec/truemotion2: Check len in tm2_read_stream()
Fixes: Timeout
Fixes: 8774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5942199639343104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
2018-07-04 12:14:00 +02:00
Michael Niedermayer
00f98d23b1 avcodec/ac3dec: Check channel_map index
Fixes: out of array read
Fixes: 8924/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EAC3_fuzzer-5851861780267008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 12:12:22 +02:00
Michael Niedermayer
bd27a9364c avcodec/mpeg4videodec: Remove use of FF_PROFILE_MPEG4_SIMPLE_STUDIO as indicator of studio profile
The profile field is changed by code inside and outside the decoder,
its not a reliable indicator of the internal codec state.
Maintaining it consistency with studio_profile is messy.
Its easier to just avoid it and use only studio_profile

Fixes: assertion failure
Fixes: ffmpeg_crash_9.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 12:11:36 +02:00
Michael Niedermayer
3b10bb8772 avcodec/shorten: Fix undefined addition in shorten_decode_frame()
Fixes: signed integer overflow: 1139785606 + 1454196085 cannot be represented in type 'int'
Fixes: 8937/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6202943597445120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Michael Niedermayer
70832333bb avcodec/shorten: Fix undefined integer overflow
Fixes: signed integer overflow: 8454144 * 256 cannot be represented in type 'int'
Fixes: 8788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-5728205041303552

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Michael Niedermayer
652d7c6348 avcodec/jpeg2000dec: Fixes invalid shifts in jpeg2000_decode_packets_po_iteration()
Fixes: shift exponent 47 is too large for 32-bit type 'int'
Fixes: 9163/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5661750182543360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Michael Niedermayer
0898a3d990 avcodec/jpeg2000dec: Check that there are enough bytes for all tiles
Fixes: OOM
Fixes: 8781/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5810709081358336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-04 01:10:28 +02:00
Rostislav Pehlivanov
755e618399 lavc: implement an ATRAC9 decoder
This commit implements a full ATRAC9 decoder, a simple low-delay codec
developed by Sony and used in most PSVita games, some PS3 games and some
PS4 games. Its similar to AAC in that it uses Huffman coded scalefactors
but instead of vector quantization it just Huffman codes the spectral
coefficients (in a way similar to how Opus splits band energy coding
into coarse and fine precision). It opts to write rather large Huffman
codes by packing several small coefficients into one Huffman coded
symbol, though I don't believe this increases efficiency at all.
Band extension implements SBC in a simple way, first it mirrors the
lower spectrum onto the higher frequencies and then it uses one of 5
filters to shape it. Noise substitution is implemented via 2 of them.
Unlike previous ATRAC codecs, there's no QMF, this is a standard MDCT
codec.

Based off of the reverse engineering work of Alex Barney.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2018-07-03 20:15:01 +01:00
Michael Niedermayer
d24c9e55f6 avcodec/dvdsubdec: Check for fully transparent rectangles earlier
Testcase with large transparent rectangles changes from 67 sec to 3 sec decode time

Fixes: Timeout
Fixes: 8728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVDSUB_fuzzer-5190088756559872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-02 13:55:28 +02:00
Michael Niedermayer
d08d4a8c73 avcodec/vc1_block: Fix mqaunt check for negative values
Fixes: out of array access
Fixes: ffmpeg_bof_4.avi
Fixes: ffmpeg_bof_5.avi
Fixes: ffmpeg_bof_6.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Reviewed-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-30 00:41:35 +02:00
Jerome Borsboom
efde2a70ff avcodec/vc1: fix check for missing CBPTAB
CBPTAB must be present in (non skipped) P and B pictures.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:19:10 +02:00
Jerome Borsboom
975a1a81b2 avcodec/vc1: fix overlap filter for frame interlaced pictures
The overlap filter is not correct for vertical edges in frame interlaced
I and P pictures. When filtering macroblocks with different FIELDTX values,
we have to match the lines at both sides of the vertical border. In addition,
we have to use the correct rounding values, depending on the line we are
filtering.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:18:44 +02:00
Jerome Borsboom
f92e95e9b5 avcodec/vc1: change the internal ordering of blocks within a macroblock
The overlap filter needs to cover a full macroblock vertical edge when the
FIELDTX value for two neighbouring macroblocks is not equal. By changing
the internal ordering of the blocks from row major to column major, we do
not need to reinterlace a FIELDTX coded macroblock before running the overlap
filter.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:17:23 +02:00
Jerome Borsboom
1cf6129d13 avcodec/vc1: fix condition guarding overlap filter on I picture
The overlap filter needs to run when PQUANT is 9 or higher, irrespective
of CONDOVER.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-29 01:16:51 +02:00
John Stebbins
08b1d1d812 h264_slice: Fix return of incomplete frames from decoder
When not using libavformat for demuxing, AVCodecContext.has_b_frames
gets set too late causing the recovery frame heuristic in h264_refs to
incorrectly flag an early frame as recovered.

This patch sets has_b_frames earlier to prevent improperly flagging the
frame as recovered.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Michael Niedermayer
2aa9047486 avcodec/mpeg4videodec: Check read profile before setting it
Fixes: null pointer dereference
Fixes: ffmpeg_crash_7.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Michael Niedermayer
00e8181bd9 avcodec/ac3_parser: Check init_get_bits8() for failure
Fixes: null pointer dereference
Fixes: ffmpeg_crash_6.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Michael Niedermayer
75a2db5524 avcodec/dpx: Check elements in 12bps planar path
Fixes: null pointer dereference
Fixes: 8946/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DPX_fuzzer-5078915222601728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-28 21:35:01 +02:00
Timo Rothenpieler
3f953379e1 avcodec/qsvenc: fix version detection on cygwin 2018-06-27 16:04:08 +02:00
Michael Niedermayer
f59c4e4391 avcodec/escape124: Fix spelling errors in comment
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-27 13:50:14 +02:00
Michael Niedermayer
71ba79702d avcodec/dvbsubdec: Compute scoretab without iterating over pixels multiple times in compute_default_clut()
Improves speed 102->2 sec
Testcase: 8655/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVBSUB_fuzzer-6277869285146624
Fixes: Timeout

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-27 13:43:39 +02:00
Michael Niedermayer
b31189881a avcodec/ra144: Fix integer overflow in ff_eval_refl()
Fixes: signed integer overflow: -4096 * -524288 cannot be represented in type 'int'
Fixes: 8650/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RA_144_fuzzer-5734816036159488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-27 13:43:39 +02:00
Lukas Rusak
7395f13df9 v4l2_m2m: decoder: fix memory leak 2018-06-27 08:38:50 +02:00
Carl Eugen Hoyos
52e8a0d96d lavc/libopenjpeg: Support GRAY10, GRAY12 and GRAY14. 2018-06-26 00:20:04 +02:00
Michael Niedermayer
78167b498f avcodec/cscd: Check output buffer size for lzo.
Fixes: Timeout
Fixes: 8665/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CSCD_fuzzer-5768442610188288

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
2018-06-25 22:58:45 +02:00
Michael Niedermayer
84bbdc255a avcodec/ffv1dec: Check state transition table
Fixes: Timeout
Fixes: 8646/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFV1_fuzzer-5649968353247232

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 22:58:45 +02:00
Michael Niedermayer
6677c98626 avcodec/escape124: Check buf_size against num_superblocks
Fixes: Timeout
Fixes: 8722/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-4843268402577408

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 22:58:45 +02:00
Michael Niedermayer
d25c945247 avcodec/h264_parser: Reduce needed history for parsing mb index
This fixes a bug/regression with very small packets
Fixes: output_file

Regression since: 0782fb6bcb32fe3ab956a99af4cc472ff81da0c2

Reported-by: Thierry Foucu <tfoucu@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 02:11:13 +02:00
Michael Niedermayer
7719b8ccc7 avcodec/magicyuv: Check bits left in flags&1 branch
Fixes: Timeout
Fixes: 8690/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-6542020913922048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-25 02:11:13 +02:00
Michael Niedermayer
540e8c2d64 avcodec/mjpegdec: Check for end of bitstream in ljpeg_decode_rgb_scan()
Fixes: Timeout
Fixes: 8648/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-5108395525799936

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-23 01:06:29 +02:00
Michael Niedermayer
a734ff4b0e libavcodec/ffv1enc: minor cosmetic fix
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-23 01:06:29 +02:00
Carl Eugen Hoyos
061e326b60 lavc/dpx: Support 10-bit packing method b (msbpad). 2018-06-21 17:41:48 +02:00
Carl Eugen Hoyos
af1e70dd66 lavc/dpx: Support 12-bit packing method b (msbpad). 2018-06-21 17:39:51 +02:00
Marton Balint
b1e0e21646 avcodec/libzvbi-teletextdec: propagate ERASE_PAGE flag for repeated subtitle page headers
This works around a libzvbi bug (a corner case in the teletext spec):

https://sourceforge.net/p/zapping/bugs/203/
https://sourceforge.net/p/zapping/patches/20/

Fixes samples/ffmpeg-bugs/trac/ticket2086/RBT_20100801_1835.ts.

Signed-off-by: Marton Balint <cus@passwd.hu>
2018-06-20 22:26:41 +02:00
Gyan Doshi
f27b02621d avcodec/libx265: apply lavc maxrate, bufsize & rc_init_occupancy
CLI options -maxrate, -bufsize and -rc_init_occupancy can now be picked
up by the x265 wrapper.  Min. rc init has to be 1001 to avoid x265
setting it to vbv-bufsize.
2018-06-19 10:25:04 +05:30
Aman Gupta
37c2cb6a68 Revert "avcodec/mediacodecdec: wait on first frame after input buffers are full"
@xyz reported a regression on his Sony Xperia Z3 Tablet Compact where
playback would intermittently fail to start, essentially deadlocking in
the decoder. Bisecting narrowed down the issue to this commit, which was
meant as an optimization but is not necessary.

This reverts commit a75bb5496ac6e7e194f1c6fd3b87f02a52e74adb.

Signed-off-by: Aman Gupta <aman@tmm1.net>
2018-06-18 11:55:23 -07:00
Thomas Guillem
513e6a30fb avcodec/videotoolboxenc: fix invalid session on iOS
Cf. comment. Restart the VT session when the APP goes from foreground to
background and vice versa.

Signed-off-by: Aman Gupta <aman@tmm1.net>
2018-06-18 11:48:05 -07:00
Thomas Guillem
9e11d27c25 avcodec/videotoolboxenc: split initialization
Split vtenc_init() into vtenc_init() (VTEncContext initialization) and
vtenc_configure_encoder() (creates the vt session).

This commit will allow to restart the vt session while encoding.

Signed-off-by: Aman Gupta <aman@tmm1.net>
2018-06-18 11:48:05 -07:00
Thomas Guillem
ce2330bdf8 avcodec/videotoolboxenc: fix mutex/cond leak in error path
The leak could happen when the vtenc_create_encoder() function failed.

Signed-off-by: Aman Gupta <aman@tmm1.net>
2018-06-18 11:48:05 -07:00
Jerome Borsboom
e85c608a46 avcodec/vc1: remove unused ff_vc1_loop_filter_iblk
Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-18 11:57:56 +02:00
Jerome Borsboom
c68f60e80a avcodec/vc1: rewrite vc1_decode_i_blocks to align with VC-1 spec
Change vc1_decode_i_blocks to use vc1_put_blocks_clamped and
ff_vc1_i_loop_filter.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-18 11:56:45 +02:00
Jerome Borsboom
4dc7ff7e10 avcodec/vc1: add Simple and Main profile to vc1_put_signed_blocks_clamped
Simple and Main Profile also need unsigned put_pixels_clamped. Add an argument
to choose between signed and unsigned put_pixels and change function name to
vc1_put_blocks_clamped.

Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
2018-06-18 11:55:26 +02:00