Bug 33208

Summary: common filenames of .so modules are not recognized (after pip3 install)
Product: Sisyphus Reporter: Ivan Zakharyaschev <imz>
Component: python3Assignee: Grigory Ustinov <grenka>
Status: CLOSED FIXED QA Contact: qa-sisyphus
Severity: normal    
Priority: P3 CC: george, glebfm, grenka, imz, vitty, vseleznv
Version: unstable   
Hardware: all   
OS: Linux   
Bug Depends on:    
Bug Blocks: 33211    
Attachments:
Description Flags
new triplet removal patch none

Description Ivan Zakharyaschev 2017-03-06 22:22:01 MSK 
python3-3.5.1-alt7

A difference between the commonly used (and recognized) filename for .so Python modules and the idea of ALT's python3's idea of it has been discovered (by george@) by trying to use a wheel-Python-package with .so modules:

pip3 install pygame --user
python3 -c 'import pygame.base'

It includes a filename of a format like this: 

*.cpython-35m-x86_64-linux-gnu.so

and is not recognized and loaded by ALT's python3. However, such filenames are generated and used also in Debian and Fedora -- here is an example from Fedora 25 for i386 and x86_64:

$ rpm -qp python3-lxml-3.6.4-1.fc25.i686.rpm -l | fgrep .so
/usr/lib/python3.5/site-packages/lxml/etree.cpython-35m-i386-linux-gnu.so
/usr/lib/python3.5/site-packages/lxml/objectify.cpython-35m-i386-linux-gnu.so
$ 

/usr/lib64/python3.5/site-packages/lxml/etree.cpython-35m-x86_64-linux-gnu.so
/usr/lib64/python3.5/site-packages/lxml/objectify.cpython-35m-x86_64-linux-gnu.so

contrary to filenames generated and recognized in ALT:

$ rpm -q python3-module-lxml -l | fgrep .so
/usr/lib64/python3/site-packages/lxml/etree.cpython-35m.so
/usr/lib64/python3/site-packages/lxml/objectify.cpython-35m.so
$ 

This could be fine, if the binaries were incompatible indeed, but they are loadable and usable after renaming (george@ checked this for pygame).

(Some of ALT's patches changed this, probably, for no reason... This is a problem for using wheels installed by pip3 without recompilation.)
Comment 1 Ivan Zakharyaschev 2017-03-06 22:28:32 MSK 
This must be determined by the following parameter -- http://stackoverflow.com/a/38525461/94687 :

$ python3
Python 3.5.1 (default, May  5 2016, 10:50:17) 
[GCC 5.3.1 20151207 (ALT Linux 5.3.1-alt3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import get_config_var
>>> get_config_var('EXT_SUFFIX')
'.cpython-35m.so'
>>>
Comment 2 Ivan Zakharyaschev 2017-03-07 15:20:27 MSK 
In Sisyphus, the fix of EXT_SUFFIX Python config parameter (bringing it to the more common format) will be convenient to make together with the move to Python 3.6. (The binary modules would anyway require a rebuild.)

(As for p8, this should be decided separately. Recognizing the common suffix in addition to ours could be a solution with little hassle; otherwise, if we change this parameter for the newly built modules, we'd need to track this with Provides/Requires to ensure both kinds of modules would be usable.)
Comment 3 Ivan Zakharyaschev 2017-03-07 17:50:27 MSK 
(In reply to comment #2)
> In Sisyphus, the fix of EXT_SUFFIX Python config parameter (bringing it to the
> more common format) will be convenient to make together with the move to Python
> 3.6. (The binary modules would anyway require a rebuild.)

This is quite clear how to do.

> (As for p8, this should be decided separately. Recognizing the common suffix in
> addition to ours could be a solution with little hassle; otherwise, if we
> change this parameter for the newly built modules, we'd need to track this with
> Provides/Requires to ensure both kinds of modules would be usable.)

This is less clear and requires hacking of a different piece of code (module finding).

Patches (and tasks for p8) are welcome!

See also: https://bugs.python.org/issue16754 which introduced the use of EXT_SUFFIX into the code.
Comment 4 Fr. Br. George 2017-07-10 23:16:39 MSK 
Created attachment 7140 [details]
new triplet removal patch

This new triplet removal patch provides both two-part and five-part EXT_SUFFIX (e. g. cpython-35m and cpython-35m-i386-linux-gnu) when searching for .so module, so 3rd-party wheels can be installed and used along with ALT native packages.
Comment 5 Fr. Br. George 2017-07-10 23:17:37 MSK 
This new triplet removal patch provides both two-part and five-part EXT_SUFFIX (e. g. cpython-35m and cpython-35m-i386-linux-gnu) when searching for .so module, so 3rd-party wheels can be installed and used along with ALT native packages.

See also task 185232
Comment 6 Grigory Ustinov 2020-02-19 19:31:47 MSK 
* Mon Jul 10 2017 Fr. Br. George <george@altlinux.ru> 3.5.1-alt9                 
- Add PLATFORM_TRIPLET suffix for binary module search