[Bug 251626] math/py-numpy: update to 1.19.4 (compatible with Python 3.9)

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Thierry Thomas <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Thierry Thomas <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|New |Open

--- Comment #22 from Thierry Thomas <***@FreeBSD.org> ---
(In reply to Antoine Brodin from comment #21)

Indeed, Mk/Uses/python.mk must be adapted:
PYNUMPY=
${PYTHON_PKGNAMEPREFIX}numpy>=1.16,1<1.20,1:math/py-numpy@${PY_FLAVOR}

@python team, what about this patch?

diff --git a/Mk/Uses/python.mk b/Mk/Uses/python.mk
index fcaf479d8076..5e2682d8efd6 100644
--- a/Mk/Uses/python.mk
+++ b/Mk/Uses/python.mk
@@ -640,7 +640,7 @@ CMAKE_ARGS+=
-DPython_ADDITIONAL_VERSIONS=${PYTHON_VER}

# Python 3rd-party modules
PYGAME=
${PYTHON_PKGNAMEPREFIX}game>0:devel/py-game@${PY_FLAVOR}
-PYNUMPY=
${PYTHON_PKGNAMEPREFIX}numpy>=1.16,1<1.20,1:math/py-numpy@${PY_FLAVOR}
+PYNUMPY=
${PYTHON_PKGNAMEPREFIX}numpy>=1.16,1<1.21,1:math/py-numpy@${PY_FLAVOR}

# Common Python modules that can be needed but only for some versions of
Python.
.if ${PYTHON_REL} < 30500

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Kubilay Kocak <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Keywords|patch |

--- Comment #23 from Kubilay Kocak <***@FreeBSD.org> ---
Shared macro's (depends lines) in python.mk are fundamentally incompatible with
the way Python dependency declarations (version-specs) work (packages can
depend individually and arbitrarily on any version of a dependency they way),
and will be removed going forward.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Kubilay Kocak <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Keywords| |needs-qa
Assignee|***@FreeBSD.org |***@FreeBSD.org
See Also| |https://github.com/freebsd/
| |pkg/issues/1911,
| |https://github.com/freebsd/
| |pkg/pull/1912

--- Comment #24 from Kubilay Kocak <***@FreeBSD.org> ---
Thank you for the update Greg. This can land and be merged with QA
confirmation.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Antoine Brodin <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|***@FreeBSD.org |***@FreeBSD.org

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Wen Heping <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|math/py-numpy: update to |math/py-numpy: update to
|1.19.4 (compatible with |1.20.3 (compatible with
|Python 3.9) |Python 3.9)

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Kubilay Kocak <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Flags|exp-run? |exp-run-
Assignee|***@FreeBSD.org |***@FreeBSD.org

--- Comment #25 from Kubilay Kocak <***@FreeBSD.org> ---
(In reply to Thierry Thomas from comment #20)

What happens/errors with autoplist?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Kubilay Kocak <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|***@FreeBSD.org |***@FreeBSD.org
Summary|math/py-numpy: update to |math/py-numpy: Update to
|1.20.3 (compatible with |1.20.3 (compatible with
|Python 3.9) |Python 3.9)

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Antoine Brodin <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Flags|exp-run- |exp-run?
Assignee|***@FreeBSD.org |***@FreeBSD.org

--- Comment #26 from Antoine Brodin <***@FreeBSD.org> ---
Exp-run is still ongoing with the additional patch Thierry provided.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #27 from Thierry Thomas <***@FreeBSD.org> ---
(In reply to Kubilay Kocak from comment #25)
It produces an uncomplete plist.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Kubilay Kocak <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|***@FreeBSD.org |***@FreeBSD.org
Keywords| |needs-patch

--- Comment #28 from Kubilay Kocak <***@FreeBSD.org> ---
^Triage: portmgr doesn't need issue assignment to run an exp-run when the flag
is requested

(In reply to Thierry Thomas from comment #27)

incomplete autoplists are almost always indicative of bugs. Can you attach a
file containing the list of whats missing? Also note that one can use autoplist
in tandem with pkg-plist (for those missing files, if we cant identify the
issue)

With the manual plist, there also appears to be some potentially problematic
entries, specifically those that need to be installed with python-version
suffixed filenames:

+bin/f2py
+bin/f2py3

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #29 from Kubilay Kocak <***@FreeBSD.org> ---
(In reply to Kubilay Kocak from comment #28)

Apologies, forgot to explicit mention that the problematic aspect is with the
use of 'concurrent'. The only package version that should install a
'suffix-less' filename is the 'default (python) version'.

I'm not sure how the semantics of this change with 'allflavors' usage, which
also begs the question, what are we using allflavors for?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #30 from Antoine Brodin <***@FreeBSD.org> ---
py37-numpy fails to package:

http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-05-31_08h30m42s/logs/errors/py37-numpy-1.20.3,1.log

New failure logs:

http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-05-31_08h30m42s/logs/errors/py38-qutip-4.6.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-05-31_08h30m42s/logs/errors/py38-geometer-0.2.3.log

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #31 from Thierry Thomas <***@FreeBSD.org> ---
(In reply to Kubilay Kocak from comment #28)

Adding autoplist to USE_PYTHON and removing pkg-plist produces this error:

$ make check-plist
====> Checking for pkg-plist issues (check-plist)
===> Parsing plist
===> Checking for items in STAGEDIR missing from pkg-plist
===> Checking for items in pkg-plist which are not in STAGEDIR
Error: Missing: %%PYTHON_SITELIBDIR%%/numpy/random/_examples/
===> Error: Plist issues found.
*** Error code 1

Stop.
make: stopped in /usr/ports/math/py-numpy

Does it work for you?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #32 from Thierry Thomas <***@FreeBSD.org> ---
(In reply to Antoine Brodin from comment #30)

1) science/py-geometer
This port does not use PYNUMPY but a custom line restricted to
numpy>=1.15,1<1.20,1

I guess that this restriction is somewhat arbitrary…
See the attached patch science_py-geometer.diff

2) devel/py-qutip
The restriction is not in the Makefile of the port, but in the setup.cfg file.
The attached patch devel_py-qutip.diff fixed the problem for me.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #33 from Thierry Thomas <***@FreeBSD.org> ---
Created attachment 225466
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=225466&action=edit
Fix science/py-geometer with a recent Numpy

Fix science/py-geometer with a recent Numpy

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #34 from Thierry Thomas <***@FreeBSD.org> ---
Created attachment 225467
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=225467&action=edit
Fix devel/py-qutip with a recent Numpy

Fix devel/py-qutip with a recent Numpy.

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #35 from Thierry Thomas <***@FreeBSD.org> ---
(In reply to Antoine Brodin from comment #30)
py37-numpy-1.20.3,1 builds OK, but the staging fails: all optimized .pyc are
missing.
Maybe someone with more Python clue could explain that and fix it?

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #36 from Kubilay Kocak <***@FreeBSD.org> ---
(In reply to Thierry Thomas from comment #31)

Interesting, thanks for that. This appears to be a common bug in upstreams
where a data_dir is added to the --record output. The best way to address this
is to remove the entry (modify the .pymodtmp pkg-plist before its written), or
fix the offending setup.py numpy/distutils issue

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #37 from Thierry Thomas <***@FreeBSD.org> ---
(In reply to Kubilay Kocak from comment #36)

Kubilay,

As much as I can admit the usage of autoplist for lazy porting of a minor port
which installs 5 or 6 files, for a port as important as Numpy, I don't
understand it , and IMHO it would be a mistake, especially if you have to patch
it in order to fix autoplist!

- a static plist allows you to simply answer the question "which port installs
this particular file?" with a grep even if not installed;

- during an upgrade, this is easy to see if some important files are missing
due to an unoticed error.

For example, the case reported by Antoine in Comment 35 (optimized files
missing with Python-3.7) would not even have been noticed with an autoplist!

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

--- Comment #38 from Antoine Brodin <***@FreeBSD.org> ---
(In reply to Thierry Thomas from comment #35)
I think that the plist issue is due to %%PYTHON_EXT_SUFFIX%% being used instead
of cpython-%%PYTHON_SUFFIX%% in pyc lines

( %%PYTHON_EXT_SUFFIX%% should only be used in .so lines )

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Thierry Thomas <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #225274|0 |1
is obsolete| |

--- Comment #39 from Thierry Thomas <***@FreeBSD.org> ---
Created attachment 225511
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=225511&action=edit
math/py-numpy: upgrade to 1.20.3

Thanks Antoine! This new patch adopted your idea.

Note: the previous plist had been generated from `make makeplist', so makeplist
must be fixed too!

[b***@freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251626

Thierry Thomas <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
URL| |https://github.com/numpy/nu
| |mpy/releases/tag/v1.20.3