Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Samuel Falvo II]

On Fri, Jul 10, 2020 at 10:13 AM
<libre-riscv-dev-request@lists.libre-riscv.org> wrote:
> Message: 4
> Date: Fri, 10 Jul 2020 17:10:17 +0100
> From: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
>
> this is why i sent you a (private) message containing the git
> repositories to clone, and pointed you at the HDL_workflow.  section
> on nmigen:
>
> mkdir ~/src
> cd !$
> git clone https://github.com/nmigen/nmigen.git
> cd nmigen
> sudo bash
> python3 setup.py develop

I don't know if you noticed, but the path name it was looking in was
underneath the libre-soc directory hierarchy, clearly being installed
contemporaneously with the other parts of the system.  That was one of
the first things I checked for.

I've tried uninstalling nmigen, and re-installing it from scratch, and
again it insisted on installing 0.1.

Just for grins, I tried with "cd nmigen && pip uninstall nmigen && pip
install -e .", and it gave me this result:

ERROR: nmigen-soc 0.0 has requirement nmigen~=0.1.rc1, but you'll have
nmigen 0.3.dev134+g30e2f91 which is incompatible.

Looks to me like there's a transitive dependency mismatch, and that is
what's causing the rollback to 0.1.

> 3. no mention of "virtualenv"

I've *ONLY* ever run nmigen in a virtualenv, and have never once had a
problem with it until now.  Something else must be the problem.  I
know it works, as my continued development of my Kestrel-3 and VDC-II
projects depend on it.

> 4. use of python3 setup.py >>>>DEVELOP<<<< *NOT* python3 setup.py >>INSTALL<<

I did this.

-- 
Samuel A. Falvo II

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Luke Kenneth Casson Leighton]

(cc'ing you direct, sam, as you're on "digest" mode.  if you make sure
to always cc yourself - and ask people at the top to cc you - it will
help people be able to reply to you much quicker, and you also won't
have to mess about cut/pasting the contents of the digest).

On Fri, Jul 10, 2020 at 9:47 PM Samuel Falvo II <sam.falvo@gmail.com> wrote:

> Just for grins, I tried with "cd nmigen && pip uninstall nmigen && pip
> install -e .", and it gave me this result:
>
> ERROR: nmigen-soc 0.0 has requirement nmigen~=0.1.rc1, but you'll have
> nmigen 0.3.dev134+g30e2f91 which is incompatible.

*click*.  that's it.  note "nmigen-soc".  you'll be able to see that
in nmigen-soc's setup.py.  (btw i cloned nmigen-soc in libre-soc.org
and *removed* that hard dependency version number.  don't use it
though because it's a hard fork of nmigen-soc).

you have a version of nmigen-soc installed (in the same virtualenv)
that DEMANDS nmigen 0.1, and exactly and ONLY nmigen 0.1.

you'll need to get rid of "nmigen-soc 0.0"... *BEFORE* you are
"permitted" to install the latest git of nmigen.

it sounds like setup.py has been modified to use pip3 by default.
that is a very worrying development and an extremely bad idea on the
part of the python developers.

samuel if you do actually use virtualenv, then so as to not disrupt
the kestrel and VDC development that you're doing, you might consider
setting up an entirely new virtualenv with a completely different
name.

i mean, it's what virtualenv is supposed to be for: create separate
completely isolated environments.  you might as well use it for that
:)

l.

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Samuel Falvo II]

On Fri, Jul 10, 2020 at 1:57 PM Luke Kenneth Casson Leighton <lkcl@lkcl.net>
wrote:
> On Fri, Jul 10, 2020 at 9:47 PM Samuel Falvo II <sam.falvo@gmail.com>
wrote:
> *click*.  that's it.  note "nmigen-soc".  you'll be able to see that
> in nmigen-soc's setup.py.  (btw i cloned nmigen-soc in libre-soc.org
> and *removed* that hard dependency version number.  don't use it
> though because it's a hard fork of nmigen-soc).
>
> you have a version of nmigen-soc installed (in the same virtualenv)
> that DEMANDS nmigen 0.1, and exactly and ONLY nmigen 0.1.

This is a completely unique virtualenv, so it shouldn't have happened.
But, what I suspect happened is I must have accidentally installed
nmigen-soc *first* before installing nmigen proper.

I've noticed (at least for me) that invoking `python setup.py develop` will
not cause an error if there's a dependency issue; `pip install -e .` will
at least issue an easily seen warning.  Running setup.py directly seems to
just make assumptions about its environment.  NOTE: I'm not suggesting one
should just stop using python setup.py develop; rather, if things seem
inexplicably askew, just try pip install -e . ; it seems to have better
diagnostics output.

I've uninstalled nmigen-soc and nmigen, then made sure to install nmigen
first, *then* nmigen-soc (both by way of setup.py develop), and it now
seems to work.  Tests are reporting back OK so far (except for skipped
tests).  (Update: now I'm getting a ton of other errors, but it's at least
made progress!)

Looking at the setup.py files in each, they seem to rely on some serious
black magic to auto-detect installed versions.  I hate stuff like that.
Configuration management is precisely where you do *NOT* want black magic
to occur (IMHO).

-- 
Samuel A. Falvo II
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Luke Kenneth Casson Leighton]

On Friday, July 10, 2020, Samuel Falvo II <sam.falvo@gmail.com> wrote:

>
> I've uninstalled nmigen-soc and nmigen, then made sure to install nmigen
> first, *then* nmigen-soc (both by way of setup.py develop), and it now
> seems to work.  Tests are reporting back OK so far (except for skipped
> tests).  (Update: now I'm getting a ton of other errors, but it's at least
> made progress!)


hurrah.

the main one you want is soc/fu/spr/test/test_pipe_caller.py

oh remember to inspect the Makefile in soc.

you want to run pywriter.py



> Looking at the setup.py files in each, they seem to rely on some serious
> black magic to auto-detect installed versions.  I hate stuff like that.
> Configuration management is precisely where you do *NOT* want black magic
> to occur (IMHO).


i know.

basically they are trying to reinvent debian distro behaviour, without the
same level of coordination and due care and attention... oh, and relying on
HTTPS for "security".

the old version of distutils is now deprecated.  it used to not have this
"intelligence".  you had to make sure everything was installed by hand.  no
dependencies were installed or "randomly searched for off the internet".

dependencies not met? error.  sort it out yourself.

l.





-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Jacob Lifshay]

apparently whitequark fixed the dependency issues with nmigen_soc:
https://github.com/nmigen/nmigen-soc/commit/5509bd2aad3012eed09c06f660cb9b60cd2cff95

So, if you install from https://github.com/nmigen/nmigen-soc instead
of PyPI or m-labs it will probably work

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Luke Kenneth Casson Leighton]

On Friday, July 10, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:

> apparently whitequark fixed the dependency issues with nmigen_soc:
> https://github.com/nmigen/nmigen-soc/commit/5509bd2aad3012eed09c06f660cb9b
> 60cd2cff95


nice.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Luke Kenneth Casson Leighton]

On Friday, July 10, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:

> apparently whitequark fixed the dependency issues with nmigen_soc:
> https://github.com/nmigen/nmigen-soc/commit/5509bd2aad3012eed09c06f660cb9b
> 60cd2cff95
>
> So, if you install from https://github.com/nmigen/nmigen-soc instead
> of PyPI or m-labs it will probably work


  install_requires=["nmigen>=0.1,<0.5"],


that will...sort-of do the trick.

installation of nmigen-soc will pull in an arbitrary, unverified and
untrusted version of nmigen off the internet

then, you git clone the nmigen repo and install that.

the pre-installed nmigen-soc dependencies will *match* the version of
nmigen being installed manually, resulting in successful replacement.

we are going to have to sort this out for the reproducible builds for
coriolis2

we absolutely cannot have arbitrary dependencies randomly downloaded, not
for any reason.

any one of those arbitrary untrusted packages could result in major
differences in the GDSII files

i mean *all* dependencies of nmigen.

pip3 is basically completely inappropriate, here, given that it could
result in USD 10 to 20 million dollars in NREs wasted because some random
software got installed.

l.



-- 
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev

Re: [libre-riscv-dev] libre-riscv-dev Digest, Vol 23, Issue 16

[Jacob Lifshay]

On Fri, Jul 10, 2020 at 4:08 PM Luke Kenneth Casson Leighton
<lkcl@lkcl.net> wrote:
>
> On Friday, July 10, 2020, Jacob Lifshay <programmerjake@gmail.com> wrote:
>
> > apparently whitequark fixed the dependency issues with nmigen_soc:
> > https://github.com/nmigen/nmigen-soc/commit/5509bd2aad3012eed09c06f660cb9b
> > 60cd2cff95
> >
> > So, if you install from https://github.com/nmigen/nmigen-soc instead
> > of PyPI or m-labs it will probably work
>
>
>   install_requires=["nmigen>=0.1,<0.5"],
>
>
> that will...sort-of do the trick.

just install nmigen first :)

Jacob

_______________________________________________
libre-riscv-dev mailing list
libre-riscv-dev@lists.libre-riscv.org
http://lists.libre-riscv.org/mailman/listinfo/libre-riscv-dev