avcodec/ffv1dec: fix some unsupported pix_fmt

When checking pix_fmt mapping, some bitstreams are mapped to an
incorrect pix_fmt instead of being rejected (ENOSYS).
Actually, such bitstreams are not supported (FFmpeg encoder does not
produce such bitstream, such bitstream may come only from another
encoder for the moment).

- JPEG 2000 RCT 11/13/15/16 bit depths are mapped to a 8-bit FFmpeg
pix_fmt (e.g. bgr0), which is not expected.
- JPEG 2000 RCT 9/10/12/14 bit depths with alpha are mapped to a
FFmpeg pix_fmt without alpha (e.g. AV_PIX_FMT_GBRP9 for 9-bit with
alpha), which is not expected.

The order for choosing the pix_fmt is changed to the one used by YCbCr
selection (<=8 bit first).
" && !f->transparency" is added to the other lines.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Jérôme Martinez 2016-06-13 19:18:22 +02:00 committed by Michael Niedermayer
parent 836f3555a8
commit 878c3a3645

View File

@ -768,17 +768,18 @@ static int read_header(FFV1Context *f)
"chroma subsampling not supported in this colorspace\n");
return AVERROR(ENOSYS);
}
if ( f->avctx->bits_per_raw_sample == 9)
if ( f->avctx->bits_per_raw_sample <= 8 && !f->transparency)
f->avctx->pix_fmt = AV_PIX_FMT_0RGB32;
else if (f->avctx->bits_per_raw_sample <= 8 && f->transparency)
f->avctx->pix_fmt = AV_PIX_FMT_RGB32;
else if (f->avctx->bits_per_raw_sample == 9 && !f->transparency)
f->avctx->pix_fmt = AV_PIX_FMT_GBRP9;
else if (f->avctx->bits_per_raw_sample == 10)
else if (f->avctx->bits_per_raw_sample == 10 && !f->transparency)
f->avctx->pix_fmt = AV_PIX_FMT_GBRP10;
else if (f->avctx->bits_per_raw_sample == 12)
else if (f->avctx->bits_per_raw_sample == 12 && !f->transparency)
f->avctx->pix_fmt = AV_PIX_FMT_GBRP12;
else if (f->avctx->bits_per_raw_sample == 14)
else if (f->avctx->bits_per_raw_sample == 14 && !f->transparency)
f->avctx->pix_fmt = AV_PIX_FMT_GBRP14;
else
if (f->transparency) f->avctx->pix_fmt = AV_PIX_FMT_RGB32;
else f->avctx->pix_fmt = AV_PIX_FMT_0RGB32;
} else {
av_log(f->avctx, AV_LOG_ERROR, "colorspace not supported\n");
return AVERROR(ENOSYS);