[libre-riscv-dev] Building Docker Containers

[libre-riscv-dev] Building Docker Containers

[Immanuel, Yehowshua U]

We really need to start building docker containers for our setups. Not saying we have to develop in docker, but be can use it as a checkpoint reference.

With my experience in software development, it can be easy to forget how you set something up if it goes unused.

Especially setting up GCCPPC which is kinda tricky - I’m quite good and fast at doing Docker - so I might be able to squeeze this in if anybody has any workflow checkpoints they’d like me to freeze.

Yehowshua
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sun, Mar 29, 2020 at 2:48 PM Immanuel, Yehowshua U
<yimmanuel3@gatech.edu> wrote:

> We really need to start building docker containers for our setups.

no.  sorry.

i'm on a mobile broadband connection.

docker FORCES massive downloads onto people, through "overlays" over
which you have absolutely no control, even on a day-by-day basis.

my brother dan asked me to install couchdb for him, in a kubernetes container.

a simple recompile - which should have been a matter of under 2
megabytes - turned into a nightmare 1 GIGABYTE download - over my
stepfather's mobile internet connection - thanks to a quotes new
release quotes of the kubernetes API from google.

he's over 70, retired, on pension, and me using up his mobile
broadband quota and forcing to have to pay a massive 40 GBP extra that
month: this is not acceptable.

so NO.

we are *not* developing code that is for the exclusive domain of the wealthy

https://www.linuxjournal.com/content/line-length-limits

"There's very little chance that Alastair's patch will make it into
the kernel. Linus Torvalds is very strict about making sure Linux
development does not favor wealthy people. He wants developers working
on ancient hardware to have the same benefits and capabilities as
those working with the benefit of the latest gadgets."

it's the same thing here.

docker is for "fixed deployment and release".  it will be
"appropriate" (not reallly) to release docckkkaaaah files once we have
stable releases.  however given the complete lack of stability,
versioning inside docker itself, and the complete and utter lack of
security in its design, you'd need to twist my arm one hell of a lot
before i *reluctantly* agree to its use for this project.

now if _someone else_ wants to do docker files, they're more than
welcome - and learn how useful they really are.  however they're
absolutely *not* going onto the FTP server: it's only a 12 GB
partition and it's 60% used already.

regarding building of gcc for powerpc: if you're doing that, there's
something wrong.  gcc is *already* available as cross-compiled debian
packages.  i have these installed:
ii  binutils-powerpc64-linux-gnu                     2.34-5
                      amd64        GNU binary utilities, for
powerpc64-linux-gnu target
ii  cpp-6-powerpc64-linux-gnu                        6.3.0-18cross1
                      amd64        GNU C preprocessor
ii  cpp-8-powerpc64-linux-gnu                        8.4.0-1cross1
                      amd64        GNU C preprocessor
ii  cpp-9-powerpc64-linux-gnu                        9.2.1-28cross1
                      amd64        GNU C preprocessor
ii  cpp-powerpc64-linux-gnu                          4:9.2.1-3
                      amd64        GNU C preprocessor (cpp) for the
ppc64 architecture
ii  g++-6-powerpc64-linux-gnu                        6.3.0-18cross1
                      amd64        GNU C++ compiler
ii  gcc-6-powerpc64-linux-gnu                        6.3.0-18cross1
                      amd64        GNU C compiler
ii  gcc-6-powerpc64-linux-gnu-base:amd64             6.3.0-18cross1
                      amd64        GCC, the GNU Compiler Collection
(base package)
ii  gcc-8-powerpc64-linux-gnu                        8.4.0-1cross1
                      amd64        GNU C compiler (cross compiler for
ppc64 architecture)
ii  gcc-8-powerpc64-linux-gnu-base:amd64             8.4.0-1cross1
                      amd64        GCC, the GNU Compiler Collection
(base package)
ii  gcc-9-powerpc64-linux-gnu                        9.2.1-28cross1
                      amd64        GNU C compiler (cross compiler for
ppc64 architecture)
ii  gcc-9-powerpc64-linux-gnu-base:amd64             9.2.1-28cross1
                      amd64        GCC, the GNU Compiler Collection
(base package)
ii  gcc-powerpc64-linux-gnu                          4:9.2.1-3
                      amd64        GNU C compiler for the ppc64
architecture

likewise qemu,

we *don't need* to build gcc for powerpc from scratch... because we're
*specifically aiming for POWER9 interoperability*.

we _will_ need a simulator, we will need llvm-ir back-end patches, and
more besides, however docker containers are the worst, most
resource-wasteful and unstable way you could possibly choose.

i take a *snapshot* of the debootstrap chroot(s) that i create
(including the pre-downloaded packages which end up in
/var/cache/apt/archives), thus saving a huge amount of bandwidth, then
run scripts in those which install (and build) the packages and
binaries needed.

this is a stable method that is also upgradeable - *both under my
control not at someone else's whim*.

ok?

please, seriously, don't drink the docker kool-aid, ok?

best,

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Immanuel, Yehowshua U]

Haha - well Georgia Tech where I am is more or less like an ISP so I have Gigabit internet when on campus - so no problems here.

I haven’t used chroot in years so I can look into that - I’ve just always had a good experience with containers for snapshots.

Yehowshua
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sun, Mar 29, 2020 at 5:03 PM Immanuel, Yehowshua U
<yimmanuel3@gatech.edu> wrote:
>
> Haha - well Georgia Tech where I am is more or less like an ISP so I have Gigabit internet when on campus - so no problems here.

yehyeh.  this is normal.  then we get someone like myself: in the
process of developing this project, just in the past two years i've
been to India, China, Taiwan, Okinawa Japan, Thailand, Hong Kong,
Spain, Belgium, Holland, England, Scotland, Canada, the U.S (briefly),
and so use

> I haven’t used chroot in years so I can look into that -

https://libre-riscv.org/HDL_workflow/coriolis2/

schroot.  cole's looking at scripting that page (it's a matter of just
cut/pasting those commands) because it's needed for the build/test
server.  do kick his tyres (yes, you cole :) ) and encourage him to
"release early, release often".  as in, "i'll wait until it's perfect
and completed" *actively* prevents and prohibits collaboration.  so if
you've got something, check it in *immediately*, ok?

(the OpenSourceEcology project actually have a rule that if you wait
more than 60 seconds before committing a file into an online
repository, that's a major red flag / violation of the collaborative
development rules)


> I’ve just always had a good experience with containers for snapshots.

snapshots.  the key word there is: snapshots.  as in, "something that
is expected to be thrown away rather than upgraded".  as in, a
*multi-gigabyte download*.... each time.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Immanuel, Yehowshua U]

> snapshots.  the key word there is: snapshots.  as in, "something that
> is expected to be thrown away rather than upgraded".  as in, a
> *multi-gigabyte download*.... each time.

Docker DOES pull in commit layers. So you can specify layers. Like a GIt layer onto of a python layer,   ontop of a ubuntu layer etc. So docker can check which layers changed and just update those - so no need for huge downloads everytime IF you specify docker layer dependencies correctly.

You usually do this in your docker file with “from ubuntu:” for “from python3”
You can also build your own layers etc…

Yehowshua

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sun, Mar 29, 2020 at 5:14 PM Luke Kenneth Casson Leighton
<lkcl@lkcl.net> wrote:
>
> On Sun, Mar 29, 2020 at 5:03 PM Immanuel, Yehowshua U
> <yimmanuel3@gatech.edu> wrote:
> >
> > Haha - well Georgia Tech where I am is more or less like an ISP so I have Gigabit internet when on campus - so no problems here.
>
> yehyeh.  this is normal.  then we get someone like myself: in the
> process of developing this project, just in the past two years i've
> been to India, China, Taiwan, Okinawa Japan, Thailand, Hong Kong,
> Spain, Belgium, Holland, England, Scotland, Canada, the U.S (briefly),
> and so use

sorry wrote the other bit and accidentally hit send.

use mobile broadband routers.  of course, for many reasons including
because i have been abroad, i don't have personal access bank account
that i can register a Contract SIM with, on credit.

therefore i use PAYG SIMS which are FIVE times more expensive.

we absolutely cannot assume that all developers, the world over, who
want to contribute to and use this project have massive bandwidth *or
even a permanent internet connection at all*.

it's why i've been pushing to get public-inbox installed, because it's
git backed.   that means people can be entirely offline and still
write messages.

the only bugbear is bugzilla, and it *might* be possible to create a
gateway between bugzilla and bugseverywhere.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sun, Mar 29, 2020 at 5:19 PM Immanuel, Yehowshua U
<yimmanuel3@gatech.edu> wrote:

> > snapshots.  the key word there is: snapshots.  as in, "something that
> > is expected to be thrown away rather than upgraded".  as in, a
> > *multi-gigabyte download*.... each time.
>
> Docker DOES pull in commit layers. So you can specify layers. Like a GIt layer onto of a python layer,   ontop of a ubuntu layer etc. So docker can check which layers changed and just update those - so no need for huge downloads everytime IF you specify docker layer dependencies correctly.
>
> You usually do this in your docker file with “from ubuntu:” for “from python3”
> You can also build your own layers etc…

yes.  not interested [and don't have time to find out what the hell is
going on].  not after the complete failure of exactly such a "layer"
setup involving google auth, kubernetes and couchdb 2.3.1 *stable*
docker release resulted in a 1.2 GB download.

i don't trust it, have never liked it, it's badly written, insecure,
and just because everyone in the world thinks it's the best thing ever
doesn't actually make that true - not to someone who's done pretty
much every major type of virtual container for linux that's existed
since 1995. (i manage a 15-year-old server that went through *three*
live conversions: XEN, to KVM, to LXC where it is, now).

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Jacob Lifshay]

I'm in the process of adding .gitlab-ci.yml files to all the repos, they
will be used for running CI on the git mirrors on debian salsa using my
build server. I'm writing them to be based on the debian:10 docker image
(basically a debian 10 minimal install) such that anyone who wants can copy
and run the commands from the .gitlab-ci.yml knowing that they are tested
on every commit. See
https://git.libre-riscv.org/?p=nmutil.git;a=blob;f=.gitlab-ci.yml;hb=HEAD
for a WIP version for nmutil.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sunday, March 29, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:

> I'm in the process of adding .gitlab-ci.yml files to all the repos, they
> will be used for running CI on the git mirrors on debian salsa using my
> build server. I'm writing them to be based on the debian:10 docker image
> (basically a debian 10 minimal install) such that anyone who wants can copy
> and run the commands from the .gitlab-ci.yml knowing that they are tested
> on every commit.


did you happen to include python3.6 and 2.7 ?

See
> https://git.libre-riscv.org/?p=nmutil.git;a=blob;f=.gitlab-ci.yml;hb=HEAD
> for a WIP version for nmutil.


great.  for test builds it's a reasonable approach.  i wouldn't personally
choose that route, we have to be careful about disk space on the talos test
server  however bandwidth there is not an issue.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Jacob Lifshay]

On Sun, Mar 29, 2020, 11:48 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Sunday, March 29, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> > I'm in the process of adding .gitlab-ci.yml files to all the repos, they
> > will be used for running CI on the git mirrors on debian salsa using my
> > build server. I'm writing them to be based on the debian:10 docker image
> > (basically a debian 10 minimal install) such that anyone who wants can
> copy
> > and run the commands from the .gitlab-ci.yml knowing that they are tested
> > on every commit.
>
>
> did you happen to include python3.6 and 2.7 ?
>

It just installs the debian python3 package, so python 3.6 I think. Python
2 can be added when we need it.

>
> See
> >
> https://git.libre-riscv.org/?p=nmutil.git;a=blob;f=.gitlab-ci.yml;hb=HEAD
> > for a WIP version for nmutil.
>
>
> great.  for test builds it's a reasonable approach.  i wouldn't personally
> choose that route, we have to be careful about disk space on the talos test
> server  however bandwidth there is not an issue.
>

My build server has a 120GB ssd, so if it works there, it should work on
the talos server.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Immanuel, Yehowshua U]

On Mar 29, 2020, at 2:59 PM, Jacob Lifshay <programmerjake@gmail.com<mailto:programmerjake@gmail.com>> wrote:

My build server has a 120GB ssd, so if it works there, it should work on
the talos server.

Careful there - you need to specify what architecture you’re using if changing to Docker PPC.
Docker by default assumes x86. Look at Raspberry Pi docker for details on how this can be done.
As I understand it, Talos is PPC non?

Yehowshua
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Jacob Lifshay]

On Sun, Mar 29, 2020, 12:33 Immanuel, Yehowshua U <yimmanuel3@gatech.edu>
wrote:

>
>
> On Mar 29, 2020, at 2:59 PM, Jacob Lifshay <programmerjake@gmail.com
> <mailto:programmerjake@gmail.com>> wrote:
>
> My build server has a 120GB ssd, so if it works there, it should work on
> the talos server.
>
> Careful there - you need to specify what architecture you’re using if
> changing to Docker PPC.
> Docker by default assumes x86. Look at Raspberry Pi docker for details on
> how this can be done.
> As I understand it, Talos is PPC non?
>

Talos is powerpc64el. Thanks for the heads-up regarding assuming x86. I was
referring to failing due to running out of disk space, which should be
nearly the same between Power and x86. I have not yet tested the gitlab CI
on the Talos, I'm trying to get it to work on x86 first. Also, since gitlab
CI is running on my local build server which I can walk over to and plug in
a keyboard and display if I need to, it might be easier to debug than if it
was running on a remote server somewhere.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sunday, March 29, 2020, Immanuel, Yehowshua U <yimmanuel3@gatech.edu>
wrote:

>
>
> On Mar 29, 2020, at 2:59 PM, Jacob Lifshay <programmerjake@gmail.com<
> mailto:programmerjake@gmail.com>> wrote:
>
> My build server has a 120GB ssd, so if it works there, it should work on
> the talos server.
>
> Careful there - you need to specify what architecture you’re using if
> changing to Docker PPC.
> Docker by default assumes x86. Look at Raspberry Pi docker for details on
> how this can be done.
> As I understand it, Talos is PPC non?


yes.

it would be good to have Makefiles and/or install.sh scripts and other
automated install things which can be called by people setting up from
scratch.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Jacob Lifshay]

On Sun, Mar 29, 2020, 12:58 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Sunday, March 29, 2020, Immanuel, Yehowshua U <yimmanuel3@gatech.edu>
> wrote:
>
> >
> >
> > On Mar 29, 2020, at 2:59 PM, Jacob Lifshay <programmerjake@gmail.com<
> > mailto:programmerjake@gmail.com>> wrote:
> >
> > My build server has a 120GB ssd, so if it works there, it should work on
> > the talos server.
> >
> > Careful there - you need to specify what architecture you’re using if
> > changing to Docker PPC.
> > Docker by default assumes x86. Look at Raspberry Pi docker for details on
> > how this can be done.
> > As I understand it, Talos is PPC non?
>
>
> yes.
>
> it would be good to have Makefiles and/or install.sh scripts and other
> automated install things which can be called by people setting up from
> scratch.
>

The gitlab CI can be set up such that it just runs the install script, if
needed. currently, the script is short enough that I don't think that's
necessary.
The install script and/or gitlab CI just runs the required make, setup.py,
and/or cargo build commands.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Cole Poirier]

On Mar 29 2020, at 1:02 pm, Jacob Lifshay <programmerjake@gmail.com> wrote:
> On Sun, Mar 29, 2020, 12:58 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> wrote:
>> it would be good to have Makefiles and/or install.sh scripts and other
>> automated install things which can be called by people setting up from
>> scratch.
> 
> The gitlab CI can be set up such that it just runs the install script, if
> needed. currently, the script is short enough that I don't think that's
> necessary.
> The install script and/or gitlab CI just runs the required make, setup.py,
> and/or cargo build commands.
> 
> Jacob

Luke, I'm doing this on the Kazan gitlab in the libre-soc-dev-env-setup
repository
(https://salsa.debian.org/Kazan-team/libre-soc-dev-env-setup). I'm
planing on making many smaller scripts that each do only one spefic
thing (eg. install yosys, install all general software prereqs from apt,
etc.). These scripts can then be combined as needed for specific
purposes/set-ups. I will write the basic use cases as these
purpose-specific aggregations, specifically the entirety of the HDL
workflow setup.

Cole

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sun, Mar 29, 2020 at 9:52 PM Cole Poirier <colepoirier@gmail.com> wrote:
>
> On Mar 29 2020, at 1:02 pm, Jacob Lifshay <programmerjake@gmail.com> wrote:
> > On Sun, Mar 29, 2020, 12:58 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> > wrote:
> >> it would be good to have Makefiles and/or install.sh scripts and other
> >> automated install things which can be called by people setting up from
> >> scratch.
> >
> > The gitlab CI can be set up such that it just runs the install script, if
> > needed. currently, the script is short enough that I don't think that's
> > necessary.
> > The install script and/or gitlab CI just runs the required make, setup.py,
> > and/or cargo build commands.
> >
> > Jacob
>
> Luke, I'm doing this on the Kazan gitlab in the libre-soc-dev-env-setup
> repository
> (https://salsa.debian.org/Kazan-team/libre-soc-dev-env-setup).

i'd very much prefer that a resource - not managed by, not backed up
by, and not with the name of our overall project - on which we will
have people being critically dependent, "go to this random third party
website on which we are guests if you want to run install scripts to
get started" - is not the first place that we direct people to.

you see how that really doesn't look like we know what we're doing?

it's absolutely fine if we were a little team with a small goal that
didn't matter in the slightest, and, particularly, if we were not
seeking USD $10m investment i'd say "sure, use external third party
hosting on which we're guest users, advertising their kindness, and
that of the hosting company rather than our own competence".

so can you please give me a name for a repository that you'd like to
be created: i will create it on git.libre-riscv.org, set you (and
anyone else) as read-writers, and we can use that, yes?

if you'd like to mirror that elsewhere, you're more than welcome to do
so (as in: nobody can stop you).  however the primary location - and
primary advertised location - really does have to be this project's
*own* services.  not a random third party one, no matter how "good"
that service is.

> I'm
> planing on making many smaller scripts that each do only one spefic
> thing (eg. install yosys, install all general software prereqs from apt,
> etc.).

that's a really good idea.  i wonder... i wonder if ikiwiki objects to
git submodules? it shouldn't....

if the scripting repo can be dropped (as a separate repo) into the
ikiwiki as a subdirectory, the scripts - and any documents - should
appear there directly as pages on the wiki.

> These scripts can then be combined as needed for specific
> purposes/set-ups. I will write the basic use cases as these
> purpose-specific aggregations, specifically the entirety of the HDL
> workflow setup.

look forward to it, as we will have interns for coriolis2 and i'd like
them to be able to get up and running with the exact same environment
in as short a time as possible.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Cole Poirier]

On Mar 29 2020, at 2:12 pm, Luke Kenneth Casson Leighton <lkcl@lkcl.net> wrote:
> On Sun, Mar 29, 2020 at 9:52 PM Cole Poirier <colepoirier@gmail.com> wrote:

>> Luke, I'm doing this on the Kazan gitlab in the libre-soc-dev-env-setup
>> repository (https://salsa.debian.org/Kazan-team/libre-soc-dev-env-setup).

> i'd very much prefer that a resource - not managed by, not backed up
> by, and not with the name of our overall project - on which we will
> have people being critically dependent, "go to this random third party
> website on which we are guests if you want to run install scripts to
> get started" - is not the first place that we direct people to.
> 
> you see how that really doesn't look like we know what we're doing?
> 
> it's absolutely fine if we were a little team with a small goal that
> didn't matter in the slightest, and, particularly, if we were not
> seeking USD $10m investment i'd say "sure, use external third party
> hosting on which we're guest users, advertising their kindness, and
> that of the hosting company rather than our own competence".
> 
> so can you please give me a name for a repository that you'd like to
> be created: i will create it on git.libre-riscv.org, set you (and
> anyone else) as read-writers, and we can use that, yes?
> 
> if you'd like to mirror that elsewhere, you're more than welcome to do
> so (as in: nobody can stop you).  however the primary location - and
> primary advertised location - really does have to be this project's
> *own* services.  not a random third party one, no matter how "good"
> that service is.

I'm with you completely Luke, I just thought to do this on gitlab
because it was on my mind while concurrently working on these env setup
scripts and the gitlab jacob-build-server setup script. I think our git
repo will be a much better place for this. Unfortunately, I'm currently
using a Debian desktop that I setup as a test, and wasn't planning on
using long term. Because of this I neglected to set it up properly using
the username that I would like to appear on my development work, so now
the name of the computer and the certs I used it to generate for my
libre-soc access are not my own. Before I step into the git repo fully,
I'd like to take this figurative milestone as an opprotunity to
reinstall my setup, and send you my new public key sometime today or
tomorrow. Is that alright luke?

>> I'm
>> planing on making many smaller scripts that each do only one spefic
>> thing (eg. install yosys, install all general software prereqs from apt,
>> etc.).
> 
> that's a really good idea.  i wonder... i wonder if ikiwiki objects to
> git submodules? it shouldn't....
> 
> if the scripting repo can be dropped (as a separate repo) into the
> ikiwiki as a subdirectory, the scripts - and any documents - should
> appear there directly as pages on the wiki.

That would certainly be very cool!

>> These scripts can then be combined as needed for specific
>> purposes/set-ups. I will write the basic use cases as these
>> purpose-specific aggregations, specifically the entirety of the HDL
>> workflow setup.
> 
> look forward to it, as we will have interns for coriolis2 and i'd like
> them to be able to get up and running with the exact same environment
> in as short a time as possible.
> 
> l.

I'm aiming for the same goal, hopefully my first draft will be ready
around tuesday.

Cole

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Sunday, March 29, 2020, Cole Poirier <colepoirier@gmail.com> wrote:

>
> libre-soc access are not my own. Before I step into the git repo fully,
> I'd like to take this figurative milestone as an opprotunity to
> reinstall my setup, and send you my new public key sometime today or
> tomorrow. Is that alright luke?


yes... except bear in mind that you in no way need to do a "Windows".

1. git --global user.name blah
2. git --global user.email blah
3. sudo bash
4. mv /home/blah /home/blahblah
5. edit /etc/passwd /etc/shadow and /etc/group

that's it.

*especially* don't throw away ssh keys please, because it will be a
sysadmin burden for me to go replacing them all with new ones, on two
separate machines.


> I'm aiming for the same goal, hopefully my first draft will be ready
> around tuesday.


great.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Jacob Lifshay]

On Mon, Mar 30, 2020, 01:59 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> yes... except bear in mind that you in no way need to do a "Windows".
>
> 1. git --global user.name blah
> 2. git --global user.email blah
> 3. sudo bash
> 4. mv /home/blah /home/blahblah
> 5. edit /etc/passwd /etc/shadow and /etc/group
>

don't forget /etc/gshadow

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Cole Poirier]

> On Mar 30, 2020, at 11:13 AM, Jacob Lifshay <programmerjake@gmail.com> wrote:
> 
> On Mon, Mar 30, 2020, 01:59 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> wrote:
> 
>> yes... except bear in mind that you in no way need to do a "Windows".
>> 
>> 1. git --global user.name blah
>> 2. git --global user.email blah
>> 3. sudo bash
>> 4. mv /home/blah /home/blahblah
>> 5. edit /etc/passwd /etc/shadow and /etc/group
>> 
> 
> don't forget /etc/gshadow
> 
> Jacob

Don’t worry I kept my ssh keys. But I was having additional problems
that I believe were self-inflicted through careless pollution of my global
package namespace among other things, so I wanted to do a clean
install so that I don’t have erroneous bugs crop up due to this during 
the HDL workflow setup. It was my mistaken assumption that changing 
my username would interfere with the proper functioning of my ssh
key because it has my old username instead of my new one. I’m happy
I was wrong :)

Cole


_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Luke Kenneth Casson Leighton]

On Mon, Mar 30, 2020 at 7:43 PM Cole Poirier <colepoirier@gmail.com> wrote:

> Don’t worry I kept my ssh keys. But I was having additional problems
> that I believe were self-inflicted through careless pollution of my global
> package namespace among other things, so I wanted to do a clean
> install so that I don’t have erroneous bugs crop up due to this during
> the HDL workflow setup.

don't do it!  teach yourself where things live, and how to find them,
and how to safely remove them, rather than assuming that it's ok to
blow away a box like it's a "toy Windoze machine" ok?

because you ain't doing that to a server, i can tell you that *right*
now!  if people are depending on a server to be up, you are *not*
going to be able to quotes blow it away quotes.

i managed once to overwrite the contents of /sbin and /usr/sbin and
other places, once, on an x86 system by unpacking a cpio-formatted ARM
initrd on it (cpio unpacks to *absolute* paths by default...)

i absolutely was *not* going to quotes reinstall it quotes because it
was a mac, which at the time was a f*****g nuisance to put UEFI
bootable linux on it.  so i rescued it by mounting then chrooting into
a live bootable install disk, unpacking some of the relevant .deb
archives (by hand) and manually copying files over, one by one.


> It was my mistaken assumption that changing
> my username would interfere with the proper functioning of my ssh
> key because it has my old username instead of my new one. I’m happy
> I was wrong :)

yes.  ssh keys have absolutely nothing to do with usernames.

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Jacob Lifshay]

On Mon, Mar 30, 2020, 11:59 Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:

> On Mon, Mar 30, 2020 at 7:43 PM Cole Poirier <colepoirier@gmail.com>
> wrote:
> > It was my mistaken assumption that changing
> > my username would interfere with the proper functioning of my ssh
> > key because it has my old username instead of my new one. I’m happy
> > I was wrong :)
>
> yes.  ssh keys have absolutely nothing to do with usernames.
>

The field in the ssh key that looks like a username is actually the comment
field, you can put anything you like there (by using one of the options for
ssh-keygen to properly edit the files), though I'm not sure if it would
need to be updated on our servers if you change the comment field. I
recommend just leaving it.

Jacob
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] Building Docker Containers

[Cole Poirier]

> On Mar 30, 2020, at 11:58 AM, Luke Kenneth Casson Leighton <lkcl@lkcl.net> wrote:
> 
> On Mon, Mar 30, 2020 at 7:43 PM Cole Poirier <colepoirier@gmail.com> wrote:
> 
>> Don’t worry I kept my ssh keys. But I was having additional problems
>> that I believe were self-inflicted through careless pollution of my global
>> package namespace among other things, so I wanted to do a clean
>> install so that I don’t have erroneous bugs crop up due to this during
>> the HDL workflow setup.
> 
> don't do it!  teach yourself where things live, and how to find them,
> and how to safely remove them, rather than assuming that it's ok to
> blow away a box like it's a "toy Windoze machine" ok?
> 
> because you ain't doing that to a server, i can tell you that *right*
> now!  if people are depending on a server to be up, you are *not*
> going to be able to quotes blow it away quotes.
> 
> i managed once to overwrite the contents of /sbin and /usr/sbin and
> other places, once, on an x86 system by unpacking a cpio-formatted ARM
> initrd on it (cpio unpacks to *absolute* paths by default...)
> 
> i absolutely was *not* going to quotes reinstall it quotes because it
> was a mac, which at the time was a f*****g nuisance to put UEFI
> bootable linux on it.  so i rescued it by mounting then chrooting into
> a live bootable install disk, unpacking some of the relevant .deb
> archives (by hand) and manually copying files over, one by one.

Don’t worry, I would never think to do this to a server, let alone one of 
yours, Jacob’s, someone else’s, or another libre-soc server. Unfortunately,
I already have done this to my local pc, but I don’t plan on doing this again. 
I had spent the past week doing the manual investigation and recovery 
method and was still encountering transient bugs and problems. If this 
were a server I would have—preferably not have been so careless 
in the first place, but barring this—continued on in my debugging, and 
reached out to the list for help if I was still unable to solve the issues 
after a second concerted effort. This is the procedure I will follow should
I unfortunately find myself with a buggy OS in the future, however I am 
planning to be much more careful and mindful about my installations 
and configurations moving forward. 

> 
>> It was my mistaken assumption that changing
>> my username would interfere with the proper functioning of my ssh
>> key because it has my old username instead of my new one. I’m happy
>> I was wrong :)
> 
> yes.  ssh keys have absolutely nothing to do with usernames.
> 
> l.

Good to know, thank you very much!

Cole


_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev