libavcodec/jpeg2000: fix precinct coordinate calculation

The calculation of precinct boundaries has been
fixed. The precinct boundaries were calculated
as an offset to the band boundary, but must
instead be calculated as an offset from the
reslevel. This patch fixes #4669 and #4679.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Gautam Ramakrishnan 2020-04-11 12:10:09 +05:30 committed by Michael Niedermayer
parent e8344f21e1
commit 995d937827

View File

@ -276,11 +276,11 @@ static int init_prec(Jpeg2000Band *band,
/* TODO: Verify with previous count of codeblocks per band */
/* Compute P_x0 */
prec->coord[0][0] = ((band->coord[0][0] >> log2_band_prec_width) + precno % reslevel->num_precincts_x) *
prec->coord[0][0] = ((reslevel->coord[0][0] >> reslevel->log2_prec_width) + precno % reslevel->num_precincts_x) *
(1 << log2_band_prec_width);
/* Compute P_y0 */
prec->coord[1][0] = ((band->coord[1][0] >> log2_band_prec_height) + precno / reslevel->num_precincts_x) *
prec->coord[1][0] = ((reslevel->coord[1][0] >> reslevel->log2_prec_height) + precno / reslevel->num_precincts_x) *
(1 << log2_band_prec_height);
/* Compute P_x1 */