[libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Jacob Lifshay]

The ieee754fpu repo is filled with tests that don't work because of
stuff like classes and imports being deleted/renamed, along with a few
files that are just syntatically incorrect. I've given up on cleaning
it up and having anything likely to still work by the time I'm done.

Luke, do you want to try to clean it up?

This is something that could have been caught before it descended into
chaos by having CI because it would fail and someone would get an
email when people break stuff, hence why I'm trying to add CI.

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Fri, Apr 3, 2020 at 8:04 AM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> The ieee754fpu repo is filled with tests that don't work because of
> stuff like classes and imports being deleted/renamed, along with a few
> files that are just syntatically incorrect. I've given up on cleaning
> it up and having anything likely to still work by the time I'm done.

commit what you have.  it doesn't surprise me because i stopped trying
to get the FSM functionality running, focussed instead on the
pipelines, and did a code reorg in the middle.

then nmigen has morphed as well and because the tests take so long i
haven't run them.

> Luke, do you want to try to clean it up?

deep breath: if it takes such a long time i am tempted to suggest that
we all tackle it (as and when).

> This is something that could have been caught before it descended into
> chaos by having CI because it would fail and someone would get an
> email when people break stuff, hence why I'm trying to add CI.

good call.

oh.

we need a CI output list.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[N.A.]

Hi,

would setting up CI and fixing the tests be eligible for receiving NLnet
funding?

Frieder

On 2020-04-03 12:24, Luke Kenneth Casson Leighton wrote:
> On Fri, Apr 3, 2020 at 8:04 AM Jacob Lifshay <programmerjake@gmail.com> wrote:
>> The ieee754fpu repo is filled with tests that don't work because of
>> stuff like classes and imports being deleted/renamed, along with a few
>> files that are just syntatically incorrect. I've given up on cleaning
>> it up and having anything likely to still work by the time I'm done.
> commit what you have.  it doesn't surprise me because i stopped trying
> to get the FSM functionality running, focussed instead on the
> pipelines, and did a code reorg in the middle.
>
> then nmigen has morphed as well and because the tests take so long i
> haven't run them.
>
>> Luke, do you want to try to clean it up?
> deep breath: if it takes such a long time i am tempted to suggest that
> we all tackle it (as and when).
>
>> This is something that could have been caught before it descended into
>> chaos by having CI because it would fail and someone would get an
>> email when people break stuff, hence why I'm trying to add CI.
> good call.
>
> oh.
>
> we need a CI output list.
>
> _______________________________________________
> libre-riscv-dev mailing list
> libre-riscv-dev@lists.libre-riscv.org
> http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Frieder Paape]

Hi,

would setting up CI and fixing the tests be eligible for receiving NLnet
funding?

Frieder

On 2020-04-03 12:24, Luke Kenneth Casson Leighton wrote:
> On Fri, Apr 3, 2020 at 8:04 AM Jacob Lifshay <programmerjake@gmail.com> wrote:
>> The ieee754fpu repo is filled with tests that don't work because of
>> stuff like classes and imports being deleted/renamed, along with a few
>> files that are just syntatically incorrect. I've given up on cleaning
>> it up and having anything likely to still work by the time I'm done.
> commit what you have.  it doesn't surprise me because i stopped trying
> to get the FSM functionality running, focussed instead on the
> pipelines, and did a code reorg in the middle.
>
> then nmigen has morphed as well and because the tests take so long i
> haven't run them.
>
>> Luke, do you want to try to clean it up?
> deep breath: if it takes such a long time i am tempted to suggest that
> we all tackle it (as and when).
>
>> This is something that could have been caught before it descended into
>> chaos by having CI because it would fail and someone would get an
>> email when people break stuff, hence why I'm trying to add CI.
> good call.
>
> oh.
>
> we need a CI output list.
>
> _______________________________________________
> libre-riscv-dev mailing list
> libre-riscv-dev@lists.libre-riscv.org
> http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Fri, Apr 3, 2020 at 11:30 AM Frieder Paape <frieder@paape.io> wrote:
>
> Hi,
>
> would setting up CI

it should be (because retrospectively we should have thought aboit it)
however it was not planned, so no, unless we write a proposal to cover
it.  this will involve specifically mentioning the fact that it is
documenting the ongoing success of the project, or something.

>and fixing the tests be eligible for receiving NLnet
> funding?

yes this one would be... except the only thing is we are almost out of
budget for the 2019-02-012 milestone covering the IEEE754 FPU.

and we cannot just take other budgets.

however what we can probably do is if other related tasks are
completed, perhaps those pay a tiny bit more generously (or, because
they were based on EUR living conditions, that may not be necessary)

regardless, we have to be extremely careful about that.

if we absolutely have to there does exist a cumulated budget from
Purism sponsorship.  it is not huge.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[N.A.]

Alright, it would finally be a chance for me to get a foot in the door
since gitlabs CI is something I know about and fixing tests will lead to
me getting an idea what the code is about.

On 2020-04-03 12:48, Luke Kenneth Casson Leighton wrote:
> On Fri, Apr 3, 2020 at 11:30 AM Frieder Paape <frieder@paape.io> wrote:
>> Hi,
>>
>> would setting up CI
> it should be (because retrospectively we should have thought aboit it)
> however it was not planned, so no, unless we write a proposal to cover
> it.  this will involve specifically mentioning the fact that it is
> documenting the ongoing success of the project, or something.
>
>> and fixing the tests be eligible for receiving NLnet
>> funding?
> yes this one would be... except the only thing is we are almost out of
> budget for the 2019-02-012 milestone covering the IEEE754 FPU.
>
> and we cannot just take other budgets.
>
> however what we can probably do is if other related tasks are
> completed, perhaps those pay a tiny bit more generously (or, because
> they were based on EUR living conditions, that may not be necessary)
>
> regardless, we have to be extremely careful about that.
>
> if we absolutely have to there does exist a cumulated budget from
> Purism sponsorship.  it is not huge.
>
> l.
>
> _______________________________________________
> libre-riscv-dev mailing list
> libre-riscv-dev@lists.libre-riscv.org
> http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Frieder Paape]

Alright, it would finally be a chance for me to get a foot in the door
since gitlabs CI is something I know about and fixing tests will lead to
me getting an idea what the code is about.

On 2020-04-03 12:48, Luke Kenneth Casson Leighton wrote:
> On Fri, Apr 3, 2020 at 11:30 AM Frieder Paape <frieder@paape.io> wrote:
>> Hi,
>>
>> would setting up CI
> it should be (because retrospectively we should have thought aboit it)
> however it was not planned, so no, unless we write a proposal to cover
> it.  this will involve specifically mentioning the fact that it is
> documenting the ongoing success of the project, or something.
>
>> and fixing the tests be eligible for receiving NLnet
>> funding?
> yes this one would be... except the only thing is we are almost out of
> budget for the 2019-02-012 milestone covering the IEEE754 FPU.
>
> and we cannot just take other budgets.
>
> however what we can probably do is if other related tasks are
> completed, perhaps those pay a tiny bit more generously (or, because
> they were based on EUR living conditions, that may not be necessary)
>
> regardless, we have to be extremely careful about that.
>
> if we absolutely have to there does exist a cumulated budget from
> Purism sponsorship.  it is not huge.
>
> l.
>
> _______________________________________________
> libre-riscv-dev mailing list
> libre-riscv-dev@lists.libre-riscv.org
> http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Fri, Apr 3, 2020 at 11:52 AM Frieder Paape <frieder@paape.io> wrote:
>
> Alright, it would finally be a chance for me to get a foot in the door
> since gitlabs CI is something I know about and fixing tests will lead to
> me getting an idea what the code is about.

ok great.

jacob you good with that?

and with frieder helping on the backend server? (will need your ssh
pubkey, frieder)

we can, say, use... EUR 500 from the Purism budget, and see how it goes?

one top priority thing, working out how we can archive copies of all
messages generated by CI

and in particular, recover the ones already generated, even if it
means writing a web scraper.

we absolutely cannot rely on salsa as an uncontrolled resource.  we
are guests on that machine and should in no way rely on that hosting
to continue to be provided (and no i am not running gitlab on the 1GB
memory allocation vCPU behind libreriscv!)

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Jacob Lifshay]

On Fri, Apr 3, 2020 at 4:10 AM Luke Kenneth Casson Leighton
<lkcl@lkcl.net> wrote:
>
> On Fri, Apr 3, 2020 at 11:52 AM Frieder Paape <frieder@paape.io> wrote:
> >
> > Alright, it would finally be a chance for me to get a foot in the door
> > since gitlabs CI is something I know about and fixing tests will lead to
> > me getting an idea what the code is about.
>
> ok great.
>
> jacob you good with that?
>
> and with frieder helping on the backend server? (will need your ssh
> pubkey, frieder)

Sure, sounds good. A warning: gitlab-runner thinks it can use all the
free disk space, so you may want to figure out how to reconfigure it
to not use all the space.

I put some manual git syncing code in /home/git-mirroring so you can
add the new gitlab instance as an additional target repo.

Instructions on https://libre-riscv.org/resources/server-setup/git-mirroring/

Do note that I configured iptables to specifically block all local
addresses from docker, so if you try to run gitlab inside a docker
container it will probably not work.

> we can, say, use... EUR 500 from the Purism budget, and see how it goes?
>
> one top priority thing, working out how we can archive copies of all
> messages generated by CI
>
> and in particular, recover the ones already generated, even if it
> means writing a web scraper.

Should be relatively easy to use gitlab's API.

> we absolutely cannot rely on salsa as an uncontrolled resource.  we
> are guests on that machine and should in no way rely on that hosting
> to continue to be provided (and no i am not running gitlab on the 1GB
> memory allocation vCPU behind libreriscv!)

Luke, do note that we are currently relying on the combination of
github and debian salsa to host Kazan's source code, so if they both
kick us out, we'd have to move all the source. We could probably add
some more mirrors using the above git-mirroring script since due to
your selection of port 922 instead of port 22 for ssh, gitlab doesn't
support push mirroring to libre-riscv.org.

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Jacob Lifshay]

On Sun, Apr 5, 2020 at 5:59 PM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> On Fri, Apr 3, 2020 at 4:10 AM Luke Kenneth Casson Leighton
> <lkcl@lkcl.net> wrote:
> > one top priority thing, working out how we can archive copies of all
> > messages generated by CI
> >
> > and in particular, recover the ones already generated, even if it
> > means writing a web scraper.
>
> Should be relatively easy to use gitlab's API.

One idea is to store all the scraped build logs in a git repo, which
makes it easy to move it around and incrementally download stuff.
We're working on doing the same thing for the mailing list.

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Mon, Apr 6, 2020 at 2:06 AM Jacob Lifshay <programmerjake@gmail.com> wrote:

> > Should be relatively easy to use gitlab's API.
>
> One idea is to store all the scraped build logs in a git repo, which
> makes it easy to move it around and incrementally download stuff.
> We're working on doing the same thing for the mailing list.

as long as it gets sent by email a list can be set up to archive from there.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Mon, Apr 6, 2020 at 1:59 AM Jacob Lifshay <programmerjake@gmail.com> wrote:

> Luke, do note that we are currently relying on the combination of
> github and debian salsa to host Kazan's source code, so if they both
> kick us out, we'd have to move all the source. We could probably add
> some more mirrors using the above git-mirroring script since due to
> your selection of port 922 instead of port 22 for ssh, gitlab doesn't
> support push mirroring to libre-riscv.org.

blech.  maybe time to move to port 22.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Frieder Paape]

Hi,

finally I found some time to look into this, I've seen that Jacob
already wrote the CI test and its working fine. My next steps would be
to mark all tests that are currently failing as skipped. And then fix
them one by one.

I am however a bit confused about my tasks because Luke says I should
give priority to the CI logging, which I believe Jacob has already taken
care of in rust. I do know the language so I could help out there as well.

Finally I have access to Jacobs build server now. I'd like to ask again
what my tasks here are. I can see if I can reconfigure gitlab-runner to
limit it's disk space. Also I understood that I should try and install
gitlab on the server?

Frieder

On 2020-04-06 02:59, Jacob Lifshay wrote:
> On Fri, Apr 3, 2020 at 4:10 AM Luke Kenneth Casson Leighton
> <lkcl@lkcl.net> wrote:
>> On Fri, Apr 3, 2020 at 11:52 AM Frieder Paape <frieder@paape.io> wrote:
>>> Alright, it would finally be a chance for me to get a foot in the door
>>> since gitlabs CI is something I know about and fixing tests will lead to
>>> me getting an idea what the code is about.
>> ok great.
>>
>> jacob you good with that?
>>
>> and with frieder helping on the backend server? (will need your ssh
>> pubkey, frieder)
> Sure, sounds good. A warning: gitlab-runner thinks it can use all the
> free disk space, so you may want to figure out how to reconfigure it
> to not use all the space.
>
> I put some manual git syncing code in /home/git-mirroring so you can
> add the new gitlab instance as an additional target repo.
>
> Instructions on https://libre-riscv.org/resources/server-setup/git-mirroring/
>
> Do note that I configured iptables to specifically block all local
> addresses from docker, so if you try to run gitlab inside a docker
> container it will probably not work.
>
>> we can, say, use... EUR 500 from the Purism budget, and see how it goes?
>>
>> one top priority thing, working out how we can archive copies of all
>> messages generated by CI
>>
>> and in particular, recover the ones already generated, even if it
>> means writing a web scraper.
> Should be relatively easy to use gitlab's API.
>
>> we absolutely cannot rely on salsa as an uncontrolled resource.  we
>> are guests on that machine and should in no way rely on that hosting
>> to continue to be provided (and no i am not running gitlab on the 1GB
>> memory allocation vCPU behind libreriscv!)
> Luke, do note that we are currently relying on the combination of
> github and debian salsa to host Kazan's source code, so if they both
> kick us out, we'd have to move all the source. We could probably add
> some more mirrors using the above git-mirroring script since due to
> your selection of port 922 instead of port 22 for ssh, gitlab doesn't
> support push mirroring to libre-riscv.org.
>
> Jacob
>
> _______________________________________________
> libre-riscv-dev mailing list
> libre-riscv-dev@lists.libre-riscv.org
> http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Sat, Apr 11, 2020 at 4:56 PM Frieder Paape <frieder@paape.io> wrote:
>
> Hi,
>
> finally I found some time to look into this, I've seen that Jacob
> already wrote the CI test and its working fine. My next steps would be
> to mark all tests that are currently failing as skipped. And then fix
> them one by one.
>
> I am however a bit confused about my tasks because Luke says I should
> give priority to the CI logging, which I believe Jacob has already taken
> care of in rust. I do know the language so I could help out there as well.

great.  the sooner we can free jacob up to help with the hardware
design on the memory subsystem the better (this is rapidly becoming a
critical path task, jacob)

> Finally I have access to Jacobs build server now. I'd like to ask again
> what my tasks here are. I can see if I can reconfigure gitlab-runner to
> limit it's disk space. Also I understood that I should try and install
> gitlab on the server?

yes please - sufficient to run ci.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Jacob Lifshay]

On Sat, Apr 11, 2020 at 8:56 AM Frieder Paape <frieder@paape.io> wrote:
>
> Hi,
>
> finally I found some time to look into this, I've seen that Jacob
> already wrote the CI test and its working fine. My next steps would be
> to mark all tests that are currently failing as skipped. And then fix
> them one by one.

the .gitlab-ci.yml file works fine, however the python code has
missing imports and missing classes which make the tests fail even if
you try to set them to skip, since they fail before invoking the test
function, which is where the skip decorator works.

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] sorry state of ieee754fpu repo -- CI desperately needed

[Luke Kenneth Casson Leighton]

On Sunday, April 12, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:

>
> the .gitlab-ci.yml file works fine, however the python code has
> missing imports and missing classes which make the tests fail even if
> you try to set them to skip, since they fail before invoking the test
> function, which is where the skip decorator works.


current highest priority is getting the core execution engine up and
running.

this is my current focus as we need to demonstrate actual ability to
execute POWER instructions very rapidly in order to engage with IBM
Engineers who are waiting on us.

having the memory subsystem up and running is becoming a high priority.

the FP code is not a high priority and will not be for about a month.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev