Bug 33208 - common filenames of .so modules are not recognized (after pip3 install)
Summary: common filenames of .so modules are not recognized (after pip3 install)
Status: CLOSED FIXED
Alias: None
Product: Sisyphus
Classification: Development
Component: python3 (show other bugs)
Version: unstable
Hardware: all Linux
: P3 normal
Assignee: Grigory Ustinov
QA Contact: qa-sisyphus
URL:
Keywords:
Depends on:
Blocks: 33211
  Show dependency tree
 
Reported: 2017-03-06 22:22 MSK by Ivan Zakharyaschev
Modified: 2020-02-19 19:31 MSK (History)
6 users (show)

See Also:


Attachments
new triplet removal patch (2.65 KB, patch)
2017-07-10 23:16 MSK, Fr. Br. George
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Zakharyaschev 2017-03-06 22:22:01 MSK
python3-3.5.1-alt7

A difference between the commonly used (and recognized) filename for .so Python modules and the idea of ALT's python3's idea of it has been discovered (by george@) by trying to use a wheel-Python-package with .so modules:

pip3 install pygame --user
python3 -c 'import pygame.base'

It includes a filename of a format like this: 

*.cpython-35m-x86_64-linux-gnu.so

and is not recognized and loaded by ALT's python3. However, such filenames are generated and used also in Debian and Fedora -- here is an example from Fedora 25 for i386 and x86_64:

$ rpm -qp python3-lxml-3.6.4-1.fc25.i686.rpm -l | fgrep .so
/usr/lib/python3.5/site-packages/lxml/etree.cpython-35m-i386-linux-gnu.so
/usr/lib/python3.5/site-packages/lxml/objectify.cpython-35m-i386-linux-gnu.so
$ 

/usr/lib64/python3.5/site-packages/lxml/etree.cpython-35m-x86_64-linux-gnu.so
/usr/lib64/python3.5/site-packages/lxml/objectify.cpython-35m-x86_64-linux-gnu.so

contrary to filenames generated and recognized in ALT:

$ rpm -q python3-module-lxml -l | fgrep .so
/usr/lib64/python3/site-packages/lxml/etree.cpython-35m.so
/usr/lib64/python3/site-packages/lxml/objectify.cpython-35m.so
$ 

This could be fine, if the binaries were incompatible indeed, but they are loadable and usable after renaming (george@ checked this for pygame).

(Some of ALT's patches changed this, probably, for no reason... This is a problem for using wheels installed by pip3 without recompilation.)
Comment 1 Ivan Zakharyaschev 2017-03-06 22:28:32 MSK
This must be determined by the following parameter -- http://stackoverflow.com/a/38525461/94687 :

$ python3
Python 3.5.1 (default, May  5 2016, 10:50:17) 
[GCC 5.3.1 20151207 (ALT Linux 5.3.1-alt3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import get_config_var
>>> get_config_var('EXT_SUFFIX')
'.cpython-35m.so'
>>>
Comment 2 Ivan Zakharyaschev 2017-03-07 15:20:27 MSK
In Sisyphus, the fix of EXT_SUFFIX Python config parameter (bringing it to the more common format) will be convenient to make together with the move to Python 3.6. (The binary modules would anyway require a rebuild.)

(As for p8, this should be decided separately. Recognizing the common suffix in addition to ours could be a solution with little hassle; otherwise, if we change this parameter for the newly built modules, we'd need to track this with Provides/Requires to ensure both kinds of modules would be usable.)
Comment 3 Ivan Zakharyaschev 2017-03-07 17:50:27 MSK
(In reply to comment #2)
> In Sisyphus, the fix of EXT_SUFFIX Python config parameter (bringing it to the
> more common format) will be convenient to make together with the move to Python
> 3.6. (The binary modules would anyway require a rebuild.)

This is quite clear how to do.

> (As for p8, this should be decided separately. Recognizing the common suffix in
> addition to ours could be a solution with little hassle; otherwise, if we
> change this parameter for the newly built modules, we'd need to track this with
> Provides/Requires to ensure both kinds of modules would be usable.)

This is less clear and requires hacking of a different piece of code (module finding).

Patches (and tasks for p8) are welcome!

See also: https://bugs.python.org/issue16754 which introduced the use of EXT_SUFFIX into the code.
Comment 4 Fr. Br. George 2017-07-10 23:16:39 MSK
Created attachment 7140 [details]
new triplet removal patch

This new triplet removal patch provides both two-part and five-part EXT_SUFFIX (e. g. cpython-35m and cpython-35m-i386-linux-gnu) when searching for .so module, so 3rd-party wheels can be installed and used along with ALT native packages.
Comment 5 Fr. Br. George 2017-07-10 23:17:37 MSK
This new triplet removal patch provides both two-part and five-part EXT_SUFFIX (e. g. cpython-35m and cpython-35m-i386-linux-gnu) when searching for .so module, so 3rd-party wheels can be installed and used along with ALT native packages.

See also task 185232
Comment 6 Grigory Ustinov 2020-02-19 19:31:47 MSK
* Mon Jul 10 2017 Fr. Br. George <george@altlinux.ru> 3.5.1-alt9                 
- Add PLATFORM_TRIPLET suffix for binary module search