[libre-riscv-dev] little-endian only power cores and spec compliance

[libre-riscv-dev] little-endian only power cores and spec compliance

[Jacob Lifshay]

Over on Libre-SOC's mailing list (CCed), we've been having a conversation
about the newly-released spec v3.1 and the compliance levels:
http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-May/006502.html

we are building a cpu that is intended to be Linux capable, however doesn't
support Power's SIMD instructions and is LE only. We've been running into
an issue with the spec's compliance levels:
the linux compliance level requires SIMD and BE is optional but the lower
compliance levels require BE but SIMD is optional.

Would it be possible to get a LE version of the int and int+float
compliance levels added to the spec?

Other ideas also welcome

Jacob Lifshay
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] little-endian only power cores and spec compliance

[Luke Kenneth Casson Leighton]

(removing opencores-hdl cc)

we need BE because without BE it is not possible to support VBLOCK,
Compressed or SVPrefix.

with BE, the opcode is presented in the first 2 bytes of any
instruction stream, regardless of whether it is 32-bit, VB, C or SVP,
and consequently the major opcodes that we plan to overload will
always be in the first 2 bytes and can reliably be used to detect the
mode.

without BE, this is flat-out impossible to do.


it would have been good to wait until we had agreed what to say on
that list because it is populated by extremely busy people whom we now
need to inform (implicitly or explicitly) that we've not discussed
properly and now their extremely precious time has to be taken up with
a correction.

can i suggest that we set some interaction guidelines for opencores-hdl?

l.

On Tue, May 12, 2020 at 8:24 AM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> Over on Libre-SOC's mailing list (CCed), we've been having a conversation
> about the newly-released spec v3.1 and the compliance levels:
> http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-May/006502.html
>
> we are building a cpu that is intended to be Linux capable, however doesn't
> support Power's SIMD instructions and is LE only. We've been running into
> an issue with the spec's compliance levels:
> the linux compliance level requires SIMD and BE is optional but the lower
> compliance levels require BE but SIMD is optional.
>
> Would it be possible to get a LE version of the int and int+float
> compliance levels added to the spec?
>
> Other ideas also welcome
>
> Jacob Lifshay
> _______________________________________________
> libre-riscv-dev mailing list
> libre-riscv-dev@lists.libre-riscv.org
> http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] little-endian only power cores and spec compliance

[Jacob Lifshay]

On Tue, May 12, 2020, 00:32 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

>
> without BE, this is flat-out impossible to do.
>

Oops, forgot about that.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] little-endian only power cores and spec compliance

[Jacob Lifshay]

On Tue, May 12, 2020, 00:40 Jacob Lifshay <programmerjake@gmail.com> wrote:

> On Tue, May 12, 2020, 00:32 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> wrote:
>
>>
>> without BE, this is flat-out impossible to do.
>>
>
> Oops, forgot about that.
>

Never mind, now that I take more than 15s to think about it, we need BE for
the instruction stream but LE for data access (since that's effectively
industry standard), so it's still effectively a LE-only architecture since
everything but the instruction decoder is LE, but the instruction decoder
can be switched to BE mode.

Having our architecture require BE for all memory accesses just because we
want to support 16-bit instructions is a non-starter in my opinion

Jacob

>
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] little-endian only power cores and spec compliance

[Luke Kenneth Casson Leighton]

On Tue, May 12, 2020 at 8:46 AM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> On Tue, May 12, 2020, 00:40 Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> > On Tue, May 12, 2020, 00:32 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> > wrote:
> >
> >>
> >> without BE, this is flat-out impossible to do.
> >>
> >
> > Oops, forgot about that.
> >
>
> Never mind, now that I take more than 15s to think about it, we need BE for
> the instruction stream but LE for data access (since that's effectively
> industry standard),

(this is a myth: only in the West is it the industry standard - see below)

> so it's still effectively a LE-only architecture since
> everything but the instruction decoder is LE, but the instruction decoder
> can be switched to BE mode.

yes.  it's.... odd.

> Having our architecture require BE for all memory accesses just because we
> want to support 16-bit instructions is a non-starter in my opinion

it's a different feature / capability.  i believe it's possible to
select any combination: BE/LE for instructions, BE/LE for data.

i can kinda see where they would want full BE support because there
still exists OSes compiled up - and maintained - to run on it.  not
supporting BE sends a clear (blunt) message to those users, "you don't
matter any more", and IBM really can't do that.

additionally, with pretty much the entirety of the Asia Industrial
complex (Japan in particular, India as well) still using BE hardware
and thus having a massive software codebase where BE is hard-coded
into control peripheral drivers, we would be cutting ourselves off
from that market by not having full BE compatibility.

the Shakti Group showed me VME Bus systems that they had to be
compatible with.  these are 40+ year old systems *still in service*,
for which radical rewrites of the software are absolutely out of the
question.

so that's a definite "decision" that we need to take seriously, rather
than dismiss lightly.

the largest (by far) downloaders of the debian ppc (not ppc64le)
distribution is Japan.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] little-endian only power cores and spec compliance

[Luke Kenneth Casson Leighton]

so, jacob: under the libre-soc charter, there's a part that says
"mistakes and merit belong to those who make it".  this means that
it's your responsibility to write to opencores-hdl-discuss to correct
the mistaken impression about BE/LE.

can i recommend:

* doing it very soon (so that they don't spend needless precious time)
* keeping it short, and
* also letting them know that we're still discussing and will get back
to them with a carefully-considered consensus response?

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] [OpenPOWER-HDL-Cores] little-endian only power cores and spec compliance

[Hugh Blemings]

Hi All,

I'm barely qualified to add much more than $0.20 to this thread, but 
will give it a whirl...

I've not had a chance to dig into the v3.1 spec as yet but can offer the 
observation that the intent (as demonstrated in the Microwatt 
implementation for example) was that the simpler/simplest 
implementations would be LE with BE optional. SIMD also optional in 
these as well.

So if the docs don't represent this we'll need to take a closer look :)

We're getting more of the key folk from our member companies (mostly 
IBM) into the loop through this list but as Luke rightly pointed out 
over on libre-riscv-dev they're a busy crew at the moment with a new 
chip due out soon :)

That said I imagine someone will chime in soon who -actually- knows this 
stuff!

Cheers,
Hugh

(Speaking for myself only in this case, though I believe wearing my ED 
hat I'd be saying much the same :)


On 12/5/20 5:24 pm, Jacob Lifshay wrote:
> Over on Libre-SOC's mailing list (CCed), we've been having a 
> conversation about the newly-released spec v3.1 and the compliance 
> levels:
> http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-May/006502.html
>
> we are building a cpu that is intended to be Linux capable, however 
> doesn't support Power's SIMD instructions and is LE only. We've been 
> running into an issue with the spec's compliance levels:
> the linux compliance level requires SIMD and BE is optional but the 
> lower compliance levels require BE but SIMD is optional.
>
> Would it be possible to get a LE version of the int and int+float 
> compliance levels added to the spec?
>
> Other ideas also welcome
>
> Jacob Lifshay
>
> _______________________________________________
> OpenPOWER-HDL-Cores mailing list
> OpenPOWER-HDL-Cores@mailinglist.openpowerfoundation.org
> http://lists.mailinglist.openpowerfoundation.org/mailman/listinfo/openpower-hdl-cores
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] [OpenPOWER-HDL-Cores] little-endian only power cores and spec compliance

[Benjamin Herrenschmidt]

On Wed, 2020-05-13 at 11:22 +1000, Hugh Blemings wrote:
> Hi All,
> I'm barely qualified to add much more than $0.20 to this thread, but
> will give it a whirl...
> I've not had a chance to dig into the v3.1 spec as yet but can offer
> the observation that the intent (as demonstrated in the Microwatt
> implementation for example) was that the simpler/simplest
> implementations would be LE with BE optional. SIMD also optional in
> these as well.

The main difficulty is that in practice, a powerpc64le toolchain will
generate code with both FP and SIMD instructions. I don't think it's
possible to generate a fully soft-float one and even if you managed
with gcc, glibc somewhat assumes it too. At least from memory.

If OpenPower wants to go down a path more similar to RiscV here and
support small(er) implementations, it's necessary to stop having the
toolchain engineers push for only supporting the latest/greatest IBM
server CPUs :-)

The Microwatt core is now starting to boot Linux as well (not merged
yet) but at the moment this requires some kernel hacks to work around
the above, such as plumbing in FPU emulation, and I know Paul has
issues generating a usable userspace...

> So if the docs don't represent this we'll need to take a closer look
> :)
> We're getting more of the key folk from our member companies (mostly
> IBM) into the loop through this list but as Luke rightly pointed out
> over on libre-riscv-dev they're a busy crew at the moment with a new
> chip due out soon :)
> That said I imagine someone will chime in soon who -actually- knows
> this stuff!
> Cheers,
> Hugh
> (Speaking for myself only in this case, though I believe wearing my
> ED hat I'd be saying much the same :)
> 
> On 12/5/20 5:24 pm, Jacob Lifshay wrote:
> > Over on Libre-SOC's mailing list (CCed), we've been having a
> > conversation about the newly-released spec v3.1 and the compliance
> > levels:
> > 
http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-May/006502.html
> > 
> > we are building a cpu that is intended to be Linux capable, however
> > doesn't support Power's SIMD instructions and is LE only. We've
> > been running into an issue with the spec's compliance levels:
> > the linux compliance level requires SIMD and BE is optional but the
> > lower compliance levels require BE but SIMD is optional.
> > 
> > Would it be possible to get a LE version of the int and int+float
> > compliance levels added to the spec?
> > 
> > Other ideas also welcome
> > 
> > Jacob Lifshay
> > 
> > 
> > _______________________________________________
> > OpenPOWER-HDL-Cores mailing list
> > OpenPOWER-HDL-Cores@mailinglist.openpowerfoundation.org
> > 
http://lists.mailinglist.openpowerfoundation.org/mailman/listinfo/openpower-hdl-cores
> 
> _______________________________________________
> OpenPOWER-HDL-Cores mailing list
> OpenPOWER-HDL-Cores@mailinglist.openpowerfoundation.org
> 
http://lists.mailinglist.openpowerfoundation.org/mailman/listinfo/openpower-hdl-cores


_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] little-endian only power cores and spec compliance

[Jacob Lifshay]

On Tue, May 12, 2020, 00:24 Jacob Lifshay <programmerjake@gmail.com> wrote:

> Over on Libre-SOC's mailing list (CCed), we've been having a conversation
> about the newly-released spec v3.1 and the compliance levels:
> http://lists.libre-riscv.org/pipermail/libre-riscv-dev/2020-May/006502.html
>
> we are building a cpu that is intended to be Linux capable, however
> doesn't support Power's SIMD instructions and is LE only. We've been
> running into an issue with the spec's compliance levels:
>

It turns out that I spoke too soon about what exactly Libre-SOC's CPU will
support, I apologise.

With that said, I still think it would be useful for OpenPower to support
LE-only processors without the additional requirements imposed by the Linux
and AIX compatibility levels.

Jacob

>
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] [OpenPOWER-HDL-Cores] little-endian only power cores and spec compliance

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Wed, May 13, 2020 at 7:01 AM Benjamin Herrenschmidt <benh@ozlabs.org> wrote:
>
> On Wed, 2020-05-13 at 11:22 +1000, Hugh Blemings wrote:
> > Hi All,
> > I'm barely qualified to add much more than $0.20 to this thread, but
> > will give it a whirl...
> > I've not had a chance to dig into the v3.1 spec as yet but can offer
> > the observation that the intent (as demonstrated in the Microwatt
> > implementation for example) was that the simpler/simplest
> > implementations would be LE with BE optional.

yes.  this was the take-away i got from reading it.  the summaries
were as follows:

* AIX Platform - POWER9 and above

   must support 64 and 32 bit
   must support LE and BE

* UNIX Platform - (i call it UNIX because RTOSes and L4Ka are not
"Linux") - *also POWER9 and above*

   must support 64 bit (32-bit probably optional, not explicity stated)
   must support LE (BE explicity stated as optional)

* SFFS - basically this recognises and gives space to the NXP QuorlQ
series (which is V2.07B(?) compliant)

  must support 32-bit (64-bit optional, explicitly stated)
  must support BE (LE explicity stated as optional)

* SFS - basically this recognises and gives space to the NXP MC5xxx
32-bit embedded series

  must support 32-bit (64-bit optional, explicitly stated)
  must support BE (LE explicity stated as optional)


note that *only POWER9* meets the requirements of the AIX and UNIX
platforms.  microwatt, by being a LE-only and non-VMX/VSX processor,
does not fall into *any* Compliance Category.

this puts microwatt firmly into "custom-built distro" territory.  our
design likewise - even though it will have 3D and VPU Vector
capability - will *still* be in "custom distro" territory, which is a
massive burden on customers and end-users.

with the product intended for mass-production, that means it'll end up
with "Rasbian" community-driven distros being created. (Rasbian was -
and is - a damn nuisance.  the original Broadcom processor used in the
Pi v1 was neither armel nor armv7: consequently it had to have its own
complete and total recompiled Debian distro, annoying the hell out of
the Debian Community due to the mis-filed bugreports and
miscategorised - unpaid - end-user support requests)

now, for Android-based products and Chromebook-based products *this is
not a problem*.  Android and chromebook products always
custom-recompile the entire binaries from source anyway.

but for distros? debian? fedora?  as things stand, we're excluded -
and so is microwatt.  microwatt as it stands cannot run *either* the
ppc32 debian distro *or* ppc64le distro binaries.

that exclusion - being unable to run standard debian or fedora distros
- in turn makes community grass-roots development almost impossible to
kick-start a thriving community and knowledgeable user-base.


> >  SIMD also optional in these as well.

unfortunately that was left out: only the SFFS and SFS subsets
(intended primarily to be 32-bit BE) are permitted to have SIMD (VMX
and VSX) optional.

it means that we (and microwatt, and any other new processor designer)
have a whopping 600 additional instructions to add (in addition to the
appx-380), with a full Vector Engine if they (we) would like their
product to take advantage of the time and effort saved by availability
of distro pre-compiled binaries (Fedora, Debian).

this is a very high threshold, which, working through the
implications, would simply have quite a significant number of
otherwise interested parties in POWER from pursuing other options.
the extra register file (Vector Regfile is separate) alone will be
enough to make them think twice.

if you already *have* VSX and VMX implemented, it is perfectly
reasonable to have it on the AIX and UNIX Platforms.  however for a
new entrant?


> The main difficulty is that in practice, a powerpc64le toolchain will
> generate code with both FP and SIMD instructions. I don't think it's
> possible to generate a fully soft-float one and even if you managed
> with gcc, glibc somewhat assumes it too. At least from memory.

a solution exists (it involves modifications to gcc, binutils and glibc).

the RISC-V Foundation members have been working on support for
multiple-compilations of functions - the exact same function -
compiled multiple times and the resultant assembly code inserted under
the *same function name* into the object file, with sufficient
metadata to be able to discern it from "functions-with-the-same-name".

(this i specifically mention because it is not a new form of c++
name-mangling being deployed, here)

what then happens at dynamic link time is this:

* the capabilities of the processor are detected
* the existence of two (or more) versions of the same dynamic-linked
function is noted
* the *CORRECT AND REQUIRED* version of that function according to the
*hardware* capabilities is selected *dynamically*.

this includes defining *new ABIs* suitable for each type of "hardware
capability".  so for example for RISC-V Vectors, they are defining a
new glibc6 ABI for vectors...

... *and compiling functions twice - once to conform to the Vector ABI
and once to conform to the standard scalar ABI*.

thus when it comes to "who must win" this is now entirely moot:
everyone can be happy.

Questions:

1) does the executable size increase?  yes of course.

2) is it the same as multilib?  honestly i am not sure that it is.
or, if it is, it's the mechanism *behind* multilib.


> If OpenPower wants to go down a path more similar to RiscV here and
> support small(er) implementations, it's necessary to stop having the
> toolchain engineers push for only supporting the latest/greatest IBM
> server CPUs :-)

i can understand why they would want to advocate that :)
unfortunately, as you point out, it excludes "simpler" entrants, and,
worse, excludes entrants to markets that *POWER9 is never going to be
able to enter anyway* due to its massive size and huge power
requirements.

however with the above scheme, recently introduced somewhere in... the
past 18+ months(?) it is no longer necessary to fight over the space.

of course, JIT-compilation by using LLVM-IR *also* solves this problem
nicely.  however, LLVM is not the default, de-facto and sole exclusive
compiler used in the world.


> The Microwatt core is now starting to boot Linux as well (not merged
> yet)

we saw the ppcdev patches.  nice work :)

> but at the moment this requires some kernel hacks to work around
> the above, such as plumbing in FPU emulation, and I know Paul has
> issues generating a usable userspace...

the powerpc gem5 experiment already has one pre-compiled (an
initramfs) with a "hello world" buildroot of some kind (busybox).  and
associated modifications to skiboot?

https://github.com/power-gem5/gem5
[branch "gem5-experimental"]

and the other support repos are at
https://github.com/power-gem5/

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev