[libre-riscv-dev] more build failures

[libre-riscv-dev] more build failures

[Jacob Lifshay]

Luke, a pile of import errors:

https://salsa.debian.org/Kazan-team/gitlab-ci-archiver/-/blob/d5411d956d4a7aefc9e6f56e75d0ae1e40175aa7/pipelines/124143/job-656506-log.txt?expanded=true&viewer=simple#L22989

I used gitlab-ci-archiver to copy the build logs to the git repo's demo
branch.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] more build failures

[Luke Kenneth Casson Leighton]

On Thu, Apr 9, 2020 at 8:21 PM Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> Luke, a pile of import errors:

yep, i am sorting out CompUnitALU, reorganising it to use decoder2.

> https://salsa.debian.org/Kazan-team/gitlab-ci-archiver/-/blob/d5411d956d4a7aefc9e6f56e75d0ae1e40175aa7/pipelines/124143/job-656506-log.txt?expanded=true&viewer=simple#L22989
>
> I used gitlab-ci-archiver to copy the build logs to the git repo's demo
> branch.

ahh, fantastic.

is that in a position to hand over to frieder or alain?

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

[libre-riscv-dev] gitlab-ci-archiver status

[Jacob Lifshay]

On Thu, Apr 9, 2020, 12:32 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Thu, Apr 9, 2020 at 8:21 PM Jacob Lifshay <programmerjake@gmail.com>
> wrote:
> > I used gitlab-ci-archiver to copy the build logs to the git repo's demo
> > branch.
>
> ahh, fantastic.
>
> is that in a position to hand over to frieder or alain?
>

Currently it needs modification to respond properly to adding a new build
to an existing pipeline (if someone clicks rebuild), it currently ignores
all pipelines already committed to the archive git repo. I'd estimate maybe
an hour of work by an experienced Rust programmer is needed.

It also needs code to send out the build failure emails, or that could be
done by a shell script by parsing all the new commits -- so an hour of work
in either bash or Rust -- both should call out to sendmail or equivalent or
we could use something like lettre.at which is a Rust email library. I
would assume the encryption referred to on lettre.at is TLS for email and
not PGP. We should probably set all emails to be forwarded through
libre-soc.org since gmail doesn't like getting emails from a home internet
ip address.

Other than those two things, it should be ready for active service.

The cron job that runs it should also run git push

It has checks to make sure the git repo is set up correctly and that it
won't mess anything up, however it won't work correctly if the repo is
being modified while it runs, so only one instance should run at a time on
each repo.

It's designed to archive the build logs of several gitlab projects at once,
so we should only need to run one instance.

The readme contains instructions for setting it up and using it:
https://salsa.debian.org/Kazan-team/gitlab-ci-archiver/-/blob/master/README.md

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Luke Kenneth Casson Leighton]

On Thu, Apr 9, 2020 at 9:03 PM Jacob Lifshay <programmerjake@gmail.com> wrote:

> > ahh, fantastic.
> >
> > is that in a position to hand over to frieder or alain?
> >
>
> Currently it needs modification to respond properly to adding a new build
> to an existing pipeline (if someone clicks rebuild), it currently ignores
> all pipelines already committed to the archive git repo. I'd estimate maybe
> an hour of work by an experienced Rust programmer is needed.

ok time's been committed, might as well finish it.

> It also needs code to send out the build failure emails, or that could be
> done by a shell script by parsing all the new commits -- so an hour of work
> in either bash or Rust

the general UNIX philosohpy is that one program do one task and do it
well.  therefore farm that task to an external program.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Jacob Lifshay]

On Thu, Apr 9, 2020, 13:25 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Thu, Apr 9, 2020 at 9:03 PM Jacob Lifshay <programmerjake@gmail.com>
> wrote:
> > It also needs code to send out the build failure emails, or that could be
> > done by a shell script by parsing all the new commits -- so an hour of
> work
> > in either bash or Rust
>
> the general UNIX philosohpy is that one program do one task and do it
> well.  therefore farm that task to an external program.
>

Ok, though both of calling out to sendmail with all the correct parameters
or writing an external script to parse the commits and then call sendmail
are likely to require much more code than just calling a library like
lettre.at even with the additional code for reading the config options from
the config file.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Luke Kenneth Casson Leighton]

On Thu, Apr 9, 2020 at 9:31 PM Jacob Lifshay <programmerjake@gmail.com> wrote:

> Ok, though both of calling out to sendmail with all the correct parameters
> or writing an external script to parse the commits and then call sendmail
> are likely to require much more code than just calling a library like
> lettre.at even with the additional code for reading the config options from
> the config file.

hm, shouldn't be: it's just exec(), right?  exec() with a list of
arguments, one of which is the tmp file that the message is saved in?

anyway if it's really that much more complicated just do it - but
please keep it expedient and rely on the existence of intermediary
servers (like the SMTP service on the VPN at 10.6.0.1, port 25)

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Jacob Lifshay]

On Thu, Apr 9, 2020, 14:48 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> anyway if it's really that much more complicated just do it - but
>
please keep it expedient and rely on the existence of intermediary
> servers (like the SMTP service on the VPN at 10.6.0.1, port 25)
>

lettre.at is specifically designed to connect to a relay server, so that
won't be an issue.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Luke Kenneth Casson Leighton]

ah i just realised, you may be tempted to assume that the outgoing command
is not 100% reliable and thus start writing retry and store and forward
code.

this is why i said assume the UNIX philosophy do one task and do it well.

other services do store and forward so use them.

we can always install exim4 as a relay.

so on consideration please use whatever wrapper rust puts around popen2 or
popen3, shove the log out on stdout, no command options (no env vars) where
the command is from the config file, no paramers, nothing.

it then becomes a 5 minute job to write a bash script "mail -s "build log"
fred@foo"

if the command does produce stderr output shove that into the error logs
and forget about it.

we need to get this handed over to alain and frieder as soon as possible.

this or equivalent will do

https://doc.rust-lang.org/1.8.0/libc/fn.popen.html

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Luke Kenneth Casson Leighton]

https://docs.rs/subprocess/0.1.18/subprocess/struct.Popen.html

this one's better.  example which is pretty much exactly what is needed
should be 5 minutes work to add and test.

https://docs.rs/subprocess/0.2.4/subprocess/

args if needed which they really shouldn't go in the array as first arg to
Popen.

anything complicated pass it to stdout in simple one line newline one line
newline form.

l.






-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Jacob Lifshay]

On Thu, Apr 9, 2020, 18:57 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> https://docs.rs/subprocess/0.1.18/subprocess/struct.Popen.html
>
> this one's better.  example which is pretty much exactly what is needed
> should be 5 minutes work to add and test.
>

I was planning on using the program invocation functionality in the Rust
standard library:
https://doc.rust-lang.org/std/process/struct.Command.html

Jacob

>
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Luke Kenneth Casson Leighton]

On Friday, April 10, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:

> On Thu, Apr 9, 2020, 18:57 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> wrote:
>
> > https://docs.rs/subprocess/0.1.18/subprocess/struct.Popen.html
> >
> > this one's better.  example which is pretty much exactly what is needed
> > should be 5 minutes work to add and test.
> >
>
> I was planning on using the program invocation functionality in the Rust
> standard library:
> https://doc.rust-lang.org/std/process/struct.Command.html


yeah that looks overkill (expect capability builtin) however the interface
looks pretty clean and the output() function has both stdout and stderr
capturable.

also the command return result (exit code) is available, which is important
to use the convention "0" as success and all nonzero values to be err
logged.

one thing is not obvious, if it is sync or async.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] gitlab-ci-archiver status

[Jacob Lifshay]

On Fri, Apr 10, 2020, 05:09 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Friday, April 10, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:
> > I was planning on using the program invocation functionality in the Rust
> > standard library:
> > https://doc.rust-lang.org/std/process/struct.Command.html
>
>
> yeah that looks overkill (expect capability builtin) however the interface
> looks pretty clean and the output() function has both stdout and stderr
> capturable.
>

The idea is to expose all the commonly used functionality in a
cross-platform manner, there are extension traits that add additional
functionality for Unix or Windows.

Unix extension trait:
https://doc.rust-lang.org/std/os/unix/process/trait.CommandExt.html

>
> also the command return result (exit code) is available, which is important
> to use the convention "0" as success and all nonzero values to be err
> logged.
>
> one thing is not obvious, if it is sync or async.
>

Depends on which functions you use if they are blocking, though it is not
async in the sense of Futures.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev