[libre-riscv-dev] submitted bugreport to upstream nmigen

[libre-riscv-dev] submitted bugreport to upstream nmigen

[Jacob Lifshay]

This is the same issue encountered when trying to add gitlab CI support.

https://github.com/nmigen/nmigen/issues/344

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

On Wednesday, April 1, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:

> This is the same issue encountered when trying to add gitlab CI support.
>
> https://github.com/nmigen/nmigen/issues/344


we may just need to write our own run_simulation function for now.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Jacob Lifshay]

On Wed, Apr 1, 2020 at 3:16 AM Luke Kenneth Casson Leighton
<lkcl@lkcl.net> wrote:
>
> On Wednesday, April 1, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> > This is the same issue encountered when trying to add gitlab CI support.
> >
> > https://github.com/nmigen/nmigen/issues/344
>
>
> we may just need to write our own run_simulation function for now.

I created a pull request that hopefully fixes this issue and
whitequark accepted it like 5 min later -- she's fast!

Rerunning nmutil CI build to find the next errors:
https://salsa.debian.org/Kazan-team/nmutil/-/jobs/642930

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

On Thu, Apr 2, 2020 at 4:04 AM Jacob Lifshay <programmerjake@gmail.com> wrote:

> > we may just need to write our own run_simulation function for now.
>
> I created a pull request that hopefully fixes this issue and
> whitequark accepted it like 5 min later -- she's fast!

fantastic to hear that. must contact her and offer some € for nmigen
work.  in particular we need simulation of SR NAND latch and support
for it to output into yosys.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Immanuel, Yehowshua U]

> fantastic to hear that. must contact her and offer some € for nmigen
> work.  in particular we need simulation of SR NAND latch and support
> for it to output into yosys.


 Yes. Whitequark is great. I had contacted ‘quark some time ago about supporting her work directly.
It would be nice if we could support her work on YosysCXX too.

Yehowshua
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Jacob Lifshay]

On Wed, Apr 1, 2020 at 8:03 PM Jacob Lifshay <programmerjake@gmail.com> wrote:
> Rerunning nmutil CI build to find the next errors:
> https://salsa.debian.org/Kazan-team/nmutil/-/jobs/642930

Fixed the remaining issues with nmutil, ended up adding
@unittest.skip() to 3 tests.

The CI build passes!

https://salsa.debian.org/Kazan-team/nmutil/-/jobs/644343

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Fri, Apr 3, 2020 at 5:10 AM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> On Wed, Apr 1, 2020 at 8:03 PM Jacob Lifshay <programmerjake@gmail.com> wrote:
> > Rerunning nmutil CI build to find the next errors:
> > https://salsa.debian.org/Kazan-team/nmutil/-/jobs/642930
>
> Fixed the remaining issues with nmutil, ended up adding
> @unittest.skip() to 3 tests.
>
> The CI build passes!

exxcellent, that's really good news.

> https://salsa.debian.org/Kazan-team/nmutil/-/jobs/644343

*pauses*.... ah.  i didn't realise that salsa - an external resource
not archived or under our direct control - had been selected for
creating information (reports) that is critically important to
preserve.

if we are kicked off of salsa or the service is terminated (debian
loses their sponsorship from gitlab) all those reports could be lost.

so we need to do three things:

1. work out how we managed to miss that, when it has been running for
such a long time already.

2. work out how we are going to archive the logs createe by Cl

3. wotk out if there is a way to get backups off of salsa of whatever
databases are being created .

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Jacob Lifshay]

On Fri, Apr 3, 2020, 03:38 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> ---
> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
>
> On Fri, Apr 3, 2020 at 5:10 AM Jacob Lifshay <programmerjake@gmail.com>
> wrote:
> >
> > On Wed, Apr 1, 2020 at 8:03 PM Jacob Lifshay <programmerjake@gmail.com>
> wrote:
> > > Rerunning nmutil CI build to find the next errors:
> > > https://salsa.debian.org/Kazan-team/nmutil/-/jobs/642930
> >
> > Fixed the remaining issues with nmutil, ended up adding
> > @unittest.skip() to 3 tests.
> >
> > The CI build passes!
>
> exxcellent, that's really good news.
>
> > https://salsa.debian.org/Kazan-team/nmutil/-/jobs/644343
>
> *pauses*.... ah.  i didn't realise that salsa - an external resource
> not archived or under our direct control - had been selected for
> creating information (reports) that is critically important to
> preserve.
>

I didn't consider it critical information since it can theoretically be
automatically regenerated (assuming we wrote our build scripts properly).

>
> if we are kicked off of salsa or the service is terminated (debian
> loses their sponsorship from gitlab) all those reports could be lost.
>
> so we need to do three things:
>
> 1. work out how we managed to miss that, when it has been running for
> such a long time already.
>
> 2. work out how we are going to archive the logs createe by Cl
>

it may not be the best solution, but there is archive.org:
https://web.archive.org/web/20200403173646/https://salsa.debian.org/Kazan-team/nmutil/-/jobs/644343/raw

>
> 3. wotk out if there is a way to get backups off of salsa of whatever
> databases are being created .


Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

On Fri, Apr 3, 2020 at 6:41 PM Jacob Lifshay <programmerjake@gmail.com> wrote:

> I didn't consider it critical information since it can theoretically be
> automatically regenerated (assuming we wrote our build scripts properly).

mmm... technically... yes.  except a year later, if we want to
precisely reproduce that exact error (regression testing), and the
salsa server is no longer available, six months prior?

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Jacob Lifshay]

On Fri, Apr 3, 2020, 11:12 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Fri, Apr 3, 2020 at 6:41 PM Jacob Lifshay <programmerjake@gmail.com>
> wrote:
>
> > I didn't consider it critical information since it can theoretically be
> > automatically regenerated (assuming we wrote our build scripts properly).
>
> mmm... technically... yes.  except a year later, if we want to
> precisely reproduce that exact error (regression testing), and the
> salsa server is no longer available, six months prior?
>

We can build it on any other gitlab installation, if all else fails, we can
install gitlab on jacob-build-server.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

On Fri, Apr 3, 2020 at 7:26 PM Jacob Lifshay <programmerjake@gmail.com> wrote:

> We can build it on any other gitlab installation, if all else fails, we can
> install gitlab on jacob-build-server.

which has ridiculous resources (32 GB RAM for a start).  plus, we have
root access so can do backups etc.

would you be happy for frieder to install gitlab on it?  (the more
people doing stuff in parallel the better).

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Jacob Lifshay]

On Fri, Apr 3, 2020 at 12:24 PM Luke Kenneth Casson Leighton
<lkcl@lkcl.net> wrote:
>
> On Fri, Apr 3, 2020 at 7:26 PM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> > We can build it on any other gitlab installation, if all else fails, we can
> > install gitlab on jacob-build-server.
>
> which has ridiculous resources (32 GB RAM for a start).  plus, we have
> root access so can do backups etc.

Actually only 20GB ram, I have 2 2GB sticks and 2 8GB sticks of ddr3
because if I have the ram why not use it. :)

If we want to get more ram, we should get ddr4 ram instead since I
also have a Ryzen 2600 and a motherboard for it that I could swap into
jacob-build-server, however I don't have any spare ddr4 ram.

> would you be happy for frieder to install gitlab on it?  (the more
> people doing stuff in parallel the better).

Sure, though it doesn't have a public IP address and will be nearly
impossible to access publicly. Also, I won't guarantee that
jacob-build-server will be up and running all the time or that I will
even respond quickly to issues (feel free to ssh in and power it off
if all else fails), so we shouldn't have it be a critical
time-sensitive dependency, such as hosting a website.

We should probably still use debian salsa for building and setup some
kind of log scraper since it is publicly visible, unlike
jacob-build-server.
I was thinking we would just put gitlab on jacob-build-server if
debian salsa is no longer viable.

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Cole Poirier]

On Apr 3 2020, at 12:24 pm, Luke Kenneth Casson Leighton <lkcl@lkcl.net> wrote:

> On Fri, Apr 3, 2020 at 7:26 PM Jacob Lifshay
> <programmerjake@gmail.com> wrote:
> 
>> We can build it on any other gitlab installation, if all else fails,
>> we can
>> install gitlab on jacob-build-server.
> 
> which has ridiculous resources (32 GB RAM for a start).  plus, we have
> root access so can do backups etc.
> 
> would you be happy for frieder to install gitlab on it?  (the more
> people doing stuff in parallel the better).
> 
> l.

I did a little bit of research on how gitlab archives build logs and
build artifacts.

https://docs.gitlab.com/ee/administration/job_artifacts.html

https://docs.gitlab.com/ee/administration/job_logs.html#new-incremental-logging-architecture

I believe this is their official tool for doing such things.

https://gitlab.com/gitlab-org/gitlab-workhorse

If you'd like I'd be willing to install gitlab and gitlab test runner on
my 2600x 32 GB machine, and purchase a sizable hard drive to serve as
one of the archives. 

Is there some way that I can allocate only 2, 4, or 6 logical cores, and
say 12 or 16GB or ram, and 50MB/s of bandwidth? Perhaps I am not yet up
to the task of running a server on which the project critically depends.
But I do have these resources available, so if there is a way you can
think of making this useful and what you require please let me know.

Cole

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

On Fri, Apr 3, 2020 at 9:22 PM Jacob Lifshay <programmerjake@gmail.com> wrote:

> > would you be happy for frieder to install gitlab on it?  (the more
> > people doing stuff in parallel the better).
>
> Sure, though it doesn't have a public IP address and will be nearly
> impossible to access publicly.

tck... tck... *thinks*.... nginx reverse-proxy through libre-riscv.org
shouuuld be able to fix that....

> Also, I won't guarantee that
> jacob-build-server will be up and running all the time or that I will
> even respond quickly to issues (feel free to ssh in and power it off
> if all else fails), so we shouldn't have it be a critical
> time-sensitive dependency, such as hosting a website.
>
> We should probably still use debian salsa for building and setup some
> kind of log scraper since it is publicly visible,

there really should be a better way of doing that.

frieder do you know how logs can be sent externally?  (cole i see you
investigated this somewhat).

remember we do *not* have administrative control over salsa.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Frieder Paape]

There is an api endpoint for jobs
<https://docs.gitlab.com/ee/api/jobs.html>. We could write a cronjob
which polls the api every day for new jobs and then saves the logs for
those jobs locally.

BUT I personaly think instead of investing time into workarounds because
we don't have gitlab hosted ourselves, we should rather invest time to
host it ourselves. Why not try to host it on the new talos 2 server.
Does it have a public IP? We will probably run into problems, but hey,
we are building a CPU on that architecture, it would look nice if we
host our development tools on such architecture as well. And if there
are larger problems we could bring them to IBM, which probably use
gitlab in high quantity and might have the muscle to ask for a quick
solution.

Hosting it on a donated x86 server I guess would be fine as well

Frieder

On 2020-04-03 23:48, Luke Kenneth Casson Leighton wrote:
> On Fri, Apr 3, 2020 at 9:22 PM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
>>> would you be happy for frieder to install gitlab on it?  (the more
>>> people doing stuff in parallel the better).
>> Sure, though it doesn't have a public IP address and will be nearly
>> impossible to access publicly.
> tck... tck... *thinks*.... nginx reverse-proxy through libre-riscv.org
> shouuuld be able to fix that....
>
>> Also, I won't guarantee that
>> jacob-build-server will be up and running all the time or that I will
>> even respond quickly to issues (feel free to ssh in and power it off
>> if all else fails), so we shouldn't have it be a critical
>> time-sensitive dependency, such as hosting a website.
>>
>> We should probably still use debian salsa for building and setup some
>> kind of log scraper since it is publicly visible,
> there really should be a better way of doing that.
>
> frieder do you know how logs can be sent externally?  (cole i see you
> investigated this somewhat).
>
> remember we do *not* have administrative control over salsa.
>
> l.
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] submitted bugreport to upstream nmigen

[Luke Kenneth Casson Leighton]

On Sat, Apr 4, 2020 at 9:15 AM Frieder Paape <frieder@paape.io> wrote:
>
> There is an api endpoint for jobs
> <https://docs.gitlab.com/ee/api/jobs.html>. We could write a cronjob
> which polls the api every day for new jobs and then saves the logs for
> those jobs locally.
>
> BUT I personaly think instead of investing time into workarounds because
> we don't have gitlab hosted ourselves, we should rather invest time to
> host it ourselves. Why not try to host it on the new talos 2 server.

although it would be a nuisance, we may need to blow it away and do a
full reinstall at the bare metal (hypervisor testing, BE OS instead of
LE OS, and so on).

> Does it have a public IP? We will probably run into problems, but hey,
> we are building a CPU on that architecture, it would look nice if we
> host our development tools on such architecture as well. And if there
> are larger problems we could bring them to IBM, which probably use
> gitlab in high quantity and might have the muscle to ask for a quick
> solution.

hmm it's good thinking: we are however reliant on them.

> Hosting it on a donated x86 server I guess would be fine as well

i pay for the mythic-beasts vCPU (and make sure it's backed up).

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev