[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[bugzilla-daemon]

http://bugs.libre-riscv.org/show_bug.cgi?id=137

--- Comment #16 from cand@gmx.com ---
# Schedule A to be attached to MoU

List of tasks, plus description, bugtracker URL and budget

# MP3 optimizations
Optimizing MP3 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=218
Budget: EUR 750

# AC3 optimizations
Optimizing AC3 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=219
Budget: EUR 750

# Vorbis optimizations
Optimizing Vorbis code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=220
Budget: EUR 750

# Opus optimizations
Optimizing Opus code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=221
Budget: EUR 750

# JPEG optimizations
Optimizing JPEG code in libjpeg-turbo with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=222
Budget: EUR 1500

# MPEG1/2 optimizations
Optimizing MPEG1/2 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=223
Budget: EUR 1500

# MPEG4 ASP optimizations
Optimizing MPEG4 ASP (xvid) code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=224
Budget: EUR 2000

# H.264 optimizations
Optimizing H.264 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=225
Budget: EUR 3000

# H.265 optimizations
Optimizing H.265 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=226
Budget: EUR 4000

# VP8 optimizations
Optimizing VP8 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=227
Budget: EUR 3000

# VP9 optimizations
Optimizing VP9 code in ffmpeg with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=228
Budget: EUR 3000

# AV1 optimizations
Optimizing AV1 code in dav1d with new instructions.

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=229
Budget: EUR 4000

# Video opcode development and discussion
Video opcode development and discussion

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=230
Budget: EUR 4000

# Video Opcodes Standards writeup
Video Opcodes Standards writeup

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=231
Budget: EUR 2000

# Implementation of video opcodes in simulator
Implementation of video opcodes in simulator

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=232
Budget: EUR 5000

# Video unit tests in simulator
Video unit tests in simulator

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=233
Budget: EUR 3000

# Hardware implementation of video opcodes
Hardware implementation of video opcodes

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=234
Budget: EUR 9000

# Video opcode FPGA tests
Video opcode FPGA tests

URL: http://bugs.libre-riscv.org/show_bug.cgi?id=235
Budget: EUR 2000

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[bugzilla-daemon]

http://bugs.libre-riscv.org/show_bug.cgi?id=137

--- Comment #17 from Luke Kenneth Casson Leighton <lkcl@lkcl.net> ---
Summary sentence for MoU

Video acceleration is a necessary component for any modern CPU and GPU. Given
the large amount of time the typical user spends on videos and its applications
like videoconferencing, a performant and power-efficient implementation is
necessary for wide adoption. With Zoom (etc.) now being critical to our modern
life, and full of security holes, full transparency in the video encode/decode
algorithms is more important than ever.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[bugzilla-daemon]

https://bugs.libre-soc.org/show_bug.cgi?id=137

--- Comment #18 from Luke Kenneth Casson Leighton <lkcl@lkcl.net> ---
status: MoU signed, sent to NLNet.  bob to countersign.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[bugzilla-daemon]

https://bugs.libre-soc.org/show_bug.cgi?id=137

Michael Pham <pham.michael.98@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pham.michael.98@gmail.com

--- Comment #19 from Michael Pham <pham.michael.98@gmail.com> ---
I see that you already sent this to NLNet, but I hope it's not too late to add
AAC to the list of Audio Formats to accelerate. The reason being is that it is
one of the primary audio formats used on YouTube and also has common use as a
Bluetooth audio format. I would even argue that AAC is even more widely used
than AC3 and Vorbis and Opus at least in my experience as an average user.

In regards to the video formats, what is the legality of putting into the
hardware to help decoding for MPEG2, MPEG4, H.264, and H.265? I know these are
at least covered by the patent pool of MPEG LA (
https://en.wikipedia.org/wiki/MPEG_LA ). Some of the audio codecs may or may
not also be tangled in legalities (not sure, I haven't really looked into it
for the audio formats).

Would it be prudent to get legal advice first? Or do you feel it is safe doing
this?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[bugzilla-daemon]

https://bugs.libre-soc.org/show_bug.cgi?id=137

--- Comment #20 from cand@gmx.com ---
I believe it's too late to add AAC, however due to how ffmpeg is structured,
some of the work on the other codecs will speed up AAC too. Not as much as
focusing on it, but more than plain sw.

MPEG2, MP3, AC3 are all free, patents expired. Vorbis and Opus were explicitly
designed for that.

For the newer video codecs, it's possible any implementation would infringe
patents. So does 90% of ffmpeg code. IANAL, but only the sw patents I believe,
as the hw blocks we will have will not be specialized to any codec. Unlike a
modern GPU that has "H.264 frame in, RGB out" blocks, we will have
sub-operations such as "calculate transform XYZ for this data". Those are not
patentable in general, some specific algorithm in hw may be.

So my conclusion is that the hw is safe, but if a commercial entity wishes to
ship our software in the US, they will need to disable the newer video codecs
or to license patents. I.e. the exact same situation as ffmpeg code, if they
want to ship that in a product with new stuff enabled.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

[libre-riscv-dev] [Bug 137] NLNet 2019 Video Acceleration Proposal 2019-10-031

[bugzilla-daemon]

https://bugs.libre-soc.org/show_bug.cgi?id=137

--- Comment #21 from Michael Pham <pham.michael.98@gmail.com> ---
(In reply to cand from comment #20)
> I believe it's too late to add AAC, however due to how ffmpeg is structured,
> some of the work on the other codecs will speed up AAC too. Not as much as
> focusing on it, but more than plain sw.
> 

Ah ok, at least it will get some treatment.

> MPEG2, MP3, AC3 are all free, patents expired. Vorbis and Opus were
> explicitly designed for that.
> 
Ok, good to hear the other audio formats are safe!

> For the newer video codecs, it's possible any implementation would infringe
> patents. So does 90% of ffmpeg code. IANAL, but only the sw patents I
> believe, as the hw blocks we will have will not be specialized to any codec.
> Unlike a modern GPU that has "H.264 frame in, RGB out" blocks, we will have
> sub-operations such as "calculate transform XYZ for this data". Those are
> not patentable in general, some specific algorithm in hw may be.
> 
> So my conclusion is that the hw is safe, but if a commercial entity wishes
> to ship our software in the US, they will need to disable the newer video
> codecs or to license patents. I.e. the exact same situation as ffmpeg code,
> if they want to ship that in a product with new stuff enabled.

I'm not a lawyer either, though your conclusion sounds reasonable to me. :)

Patents are a minefield to navigate (sigh)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev