Replacing Multiplication With Sign Flips in Video Coding A Cheaper Way to Run Optical Flow Prediction

This application covers a way of building the predicted picture in a video codec that uses bi-directional optical flow, a refinement step that adjusts each pixel of a prediction using the picture gradients of two reference blocks. The claims define five running totals that the encoder or decoder must compute, and they are written so that the awkward multiplications are replaced by simple sign changes. It was filed by Panasonic Intellectual Property Corporation of America, names Tadamasa Toma, Takahiro Nishi, Kiyofumi Abe and Yusuke Kato as inventors, and is a divisional of Australian application 2020298425. The title is the generic “Encoder, Decoder, Encoding Method, and Decoding Method”, but claim 1 is much narrower than that and is about this one calculation.

The Problem

Modern video compression works by predicting each block of a picture from blocks that have already been coded, then sending only the difference. The better the prediction, the fewer bits the difference needs. The specification starts from the earlier H.265/HEVC standard, which it cites as its one non-patent reference, and its codec history also lists H.266/VVC. Its stated aim is generic: more coding efficiency, better image quality, less processing and smaller circuits.

Bi-directional optical flow, which the specification calls BIO or BDOF, is one of the tools that gets there. A block is predicted from two reference pictures, one earlier and one later, and a per-pixel correction is then worked out from the pixel values and the horizontal and vertical gradients of those two references. The specification says this makes a highly accurate prediction image likely, which in turn is likely to reduce the amount of code in the resulting video.

The catch, in the document’s own words, is that the parameters used in the optical flow are derived from the pixel value and gradient at each pixel position in the reference image. Doing that per pixel adds computation, and that in turn may increase the circuit scale. Multiplications are the expensive part. The specification calls them “substantial multiplication”, meaning multiplying two general values together, as opposed to flipping a sign or adding.

What This Invention Does

Claim 1 is an encoder with a memory and a processor. The processor generates a prediction image for the current block from five parameters derived in a BDOF process. Each parameter is a sum over a set of relative pixel positions, written as the Greek capital omega in the claim, and each position is looked up in two gradient images and two interpolated images that correspond to the current block.

In plain terms, the five parameters are as follows. The first is the sum of the absolute value of (horizontal gradient in the first reference plus horizontal gradient in the second). The second is the same thing for the vertical gradients. The third takes the pixel difference between the two interpolated images and, for each position, flips its sign according to the sign of the summed horizontal gradient, then adds these up. The fourth does the same using the vertical gradient sign. The fifth is built from the sign of the summed vertical gradient applied to the summed horizontal gradient. The claim defines a sign function that returns -1, 0 or 1, and an abs function, and nothing in these expressions multiplies two full-size values together.

Claim 2 is the matching decoder and claim 19 is a decoding method, with further method claims. The specification explains the benefit directly. Because only plus and minus signs change in the relevant expressions, “the number of times of substantial multiplication in BIO processing can be reduced significantly”. It also states that this holds even without the step of comparing the horizontal and vertical sums, and that the same processes can be read across from decoding to encoding.

The specification is candid about the trade. It describes another specific example in which genuine multiplication is used to compute four of the totals, and says that this increases the amount of computation but generates a prediction sample with higher accuracy, while the sign-based example is the low-computation option. It gives no bit-rate savings, timing figures or gate counts for either, so the size of the gain is not quantified in the text.

One drafting point a careful reader will notice: the printed claim 1 uses the label Ix0 twice, first for the horizontal gradient value in the first gradient image and again at its close for the right-minus-left sample difference that defines how the gradient is taken. It looks like a labelling slip in the claim as filed. The printed mathematical expressions in the PDF are also partly garbled in text extraction, so the summaries above follow the claim wording and the specification’s own description of the parameters.

Key Features

• Five sums, no general multiplications. The first and second parameters are sums of absolute gradient sums, and the third to fifth are sums of values whose sign has been inverted or kept, which avoids multiplying two arbitrary quantities.
• Two reference blocks, one set of positions. Gradients and interpolated pixel values are read from a first and a second image at the same relative pixel positions, matching how bi-directional prediction takes one block from each side of the current picture.
• Sign function instead of a product. The sign function outputs -1, 0 or 1, so applying it to a pixel difference is a negate-or-keep decision instead of a multiplier.
• Mirrored claim set. The same calculation is claimed for an encoder, a decoder, an encoding method and a decoding method, which is what a codec standard needs because both ends must compute identical predictions.
• Documented accuracy option. A second example in the description trades extra multiplication for a more accurate prediction, showing that the sign-flip version is a deliberate low-cost variant and not the only way to do it.

Who Is Behind It

The applicant is Panasonic Intellectual Property Corporation of America, a United States entity of the Panasonic group, whose parent company Panasonic Holdings is the Osaka-based electronics group founded by Konosuke Matsushita in 1918, as recounted on its Wikipedia page. I could not find a standalone corporate web page for the American subsidiary itself. A patent-statistics site, Patentbots, lists 331 US patents issued to it in the year ending the first quarter of 2026, so it is a high-volume filer, mostly in video and imaging classes. The Australian agent is Griffith Hack of Melbourne.

The four named inventors are individuals; the specification gives no biographical detail about them and none is claimed here.

On priority, the specification is silent. The title page records only that this is a divisional of Australian application 2020298425, and the text contains no cross-reference or convention claim paragraph. No priority chain is asserted here. The Priority Country entry in the table below is therefore “Not stated”.

Why It Matters

Video is a huge share of internet traffic, and every new codec generation squeezes out bits by adding prediction tools that cost more computation. Versatile Video Coding, finalised in July 2020, is the standard that H.266 refers to. Whether a tool makes it into a phone or television chip depends on how much silicon and power it needs, since the decoder has to run it for every block of every frame.

That is the reason to file a claim like this one. Taking multiplications out of the inner loop of a per-pixel refinement step makes the tool cheaper to build into hardware. Because both encoder and decoder must compute exactly the same prediction, the arithmetic is effectively fixed once chosen, so a company that holds a claim on a particular cheap formulation may hold a claim that an implementer cannot easily design around.

The honest limits: the specification reports no measured compression result, and the claims are confined to one set of five defined sums. Whether this formulation ends up in any standard or product is not something the document says.

Related Concepts

• Optical flow – the idea of estimating how image content moves between frames, from which the per-pixel correction in BDOF is derived.
• Motion compensation – predicting a block from an earlier or later reference frame, the stage BDOF refines.
• High Efficiency Video Coding – the H.265 standard the specification cites as the existing baseline.
• Versatile Video Coding – the H.266 successor standard, in which this kind of tool is relevant.
• Advanced Video Coding – the earlier H.264 standard the specification lists in its codec lineage.

---

AU 2026202242 was published in the Australian Official Journal of Patents on 9 April 2026 and is open for public inspection. Patent applications represent inventions that are sought to be protected and do not necessarily reflect commercially available products.