Bug 31576 - the filetrigger works too long when installing ghc7.6.1 in a minimal system
Summary: the filetrigger works too long when installing ghc7.6.1 in a minimal system
Status: CLOSED FIXED
Alias: None
Product: Sisyphus
Classification: Development
Component: ghc7.6.1 (show other bugs)
Version: unstable
Hardware: all Linux
: P3 normal
Assignee: Denis Smirnov
QA Contact: qa-sisyphus
URL: http://git.altlinux.org/tasks/153832/...
Keywords:
Depends on:
Blocks:
 
Reported: 2015-12-01 17:49 MSK by Ivan Zakharyaschev
Modified: 2015-12-02 04:14 MSK (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Zakharyaschev 2015-12-01 17:49:49 MSK
https://lists.altlinux.org/pipermail/devel/2015-November/200457.html

The preinstall check hangs on x86_64 only (not on i586!) for Haskell libraries.

This is reproducible both for the last tag from sisyphus branch of ghc7.6.1-language-c, and from the test branch (with the old release). Can be done --test-only (if the release has been increased).

The tests on other systems (under root with APT and with hasher) don't reproduce this.
Comment 1 Ivan Zakharyaschev 2015-12-01 20:33:14 MSK
Actually, the log shows that the check on i586 also took very long (45 min).

Reproducible with hasher (thx to ldv@ for advices) -- the important thing is that hsh --initroot-only is used with non-default (more minimal) configuration:

hsh --init  --without-stuff --no-contents-ind --pkg-init-list=+altlinux-release-sisyphus --pkg-build-list=altlinux-release-sisyphus,basesystem && hsh-install ghc7.6.1

I've taken the options from http://git.altlinux.org/people/ldv/packages/?p=girar.git;a=blob;f=gb/remote/gb-remote-check-install;h=e7823af17cdfc68369f5782a8cdbac18e581adb7;hb=1535653ca9923ea8cd228ae68b2ca7c1b80eba7e#l41 .

$ apt-repo --hsh-apt-config=/home/imz/.hasher/sisyphus/apt.conf
Info: Will use hasher when appropriate (with --apt-config=/home/imz/.hasher/sisyphus/apt.conf). at /home/imz/bin/apt-repo line 30.
Info: Will try to read a non-system APT_CONFIG=/home/imz/.hasher/sisyphus/apt.conf at /home/imz/bin/apt-repo line 51.
grep: /var/empty/*.list: Нет такого файла или каталога
rpm [alt] file:/ALT/Sisyphus x86_64 classic
rpm [alt] file:/ALT/Sisyphus noarch classic
$
Comment 2 Ivan Zakharyaschev 2015-12-01 20:35:24 MSK
The problem is that ghc-pkg recache works too much.

The output when it is reproduced:

[imz@basalt ~]$ hsh --init  --without-stuff --no-contents-ind --pkg-init-list=+altlinux-release-sisyphus --pkg-build-list=altlinux-release-sisyphus,basesystem && hsh-install ghc7.6.1
<86>Dec  1 17:19:08 userdel[2182]: delete user 'rooter'
<86>Dec  1 17:19:08 groupadd[2187]: group added to /etc/group: name=rooter, GID=562
<86>Dec  1 17:19:08 groupadd[2187]: group added to /etc/gshadow: name=rooter
<86>Dec  1 17:19:08 groupadd[2187]: new group: name=rooter, GID=562
<86>Dec  1 17:19:08 useradd[2191]: new user: name=rooter, UID=562, GID=562, home=/root, shell=/bin/bash
<86>Dec  1 17:19:08 userdel[2197]: delete user 'builder'
<86>Dec  1 17:19:08 groupadd[2205]: group added to /etc/group: name=builder, GID=563
<86>Dec  1 17:19:08 groupadd[2205]: group added to /etc/gshadow: name=builder
<86>Dec  1 17:19:08 groupadd[2205]: new group: name=builder, GID=563
<86>Dec  1 17:19:08 useradd[2211]: new user: name=builder, UID=563, GID=563, home=/usr/src, shell=/bin/bash
<13>Dec  1 17:19:09 rpmi: libffi6-1:3.1-alt2 1409163098 installed
<13>Dec  1 17:19:09 rpmi: rpm-macros-ghc7.6.1-common-7.6.1-alt6 1445385767 installed
<13>Dec  1 17:19:09 rpmi: rpm-build-haskell-1-alt26 1445536638 installed
<13>Dec  1 17:19:09 rpmi: haskell-filetrigger-0.0.5-alt2 1445446241 installed
<13>Dec  1 17:19:09 rpmi: libgmp-devel-6.0.0a-alt1 1431516850 installed
<13>Dec  1 17:19:10 rpmi: glib2-locales-2.46.2-alt1 1446853826 installed
<13>Dec  1 17:19:10 rpmi: glib2-2.46.2-alt1 1446853786 installed
<13>Dec  1 17:19:10 rpmi: pkg-config-0.25-alt2 1298849081 installed
<13>Dec  1 17:19:10 rpmi: libffi-devel-1:3.1-alt2 1409163098 installed
<13>Dec  1 17:19:16 rpmi: ghc7.6.1-7.6.1-alt6 1445386600 installed
<13>Dec  1 17:19:16 rpmi: ghc7.6.1-common-7.6.1-alt6 1445385767 installed
^Chsh-install: Packages installation failed.
[imz@basalt ~]$ ^C
Comment 3 Ivan Zakharyaschev 2015-12-01 21:05:44 MSK
Now the ghc package wouldn't pass the install check of Sisyphus:

hsh --apt-config=/home/imz/.hasher/sisyphus/apt.conf --init  --without-stuff --no-contents-ind --pkg-init-list=+altlinux-release-sisyphus --pkg-build-list=altlinux-release-sisyphus,basesystem && hsh-install ghc7.6.1- ghc7.6.1 && hsh-install ghc7.6.1

This is because rpm-build-haskell-1-alt26 lost its dependency on rpm-build in the meantime (after ghc was built for Sisyphus the last time).
Comment 4 Ivan Zakharyaschev 2015-12-01 21:07:38 MSK
Now the ghc package wouldn't pass the install check of Sisyphus:

hsh --apt-config=/home/imz/.hasher/sisyphus/apt.conf --init  --without-stuff
--no-contents-ind --pkg-init-list=+altlinux-release-sisyphus
--pkg-build-list=altlinux-release-sisyphus,basesystem && hsh-install ghc7.6.1-
ghc7.6.1 && hsh-install ghc7.6.1

This is because rpm-build-haskell-1-alt26 lost its dependency on rpm-build in
the meantime (after ghc was built for Sisyphus the last time).
Comment 5 Ivan Zakharyaschev 2015-12-01 21:35:11 MSK
Adding gcc to the environment makes it work:

hsh-install ghc7.6.1- ghc7.6.1 gcc && hsh-install ghc7.6.1
Comment 6 Ivan Zakharyaschev 2015-12-01 22:00:57 MSK
I was mistaken about the importance of gcc for this process.

The thing that make the difference is glibc-gconv-modules-6:2.17-alt8 .

If it is present, it works; if not, it hangs (works too much).

I've found out this by doing:

rpm -Uhv --notriggers /var/cache/apt/archives/ghc7.6.1-common_7.6.1-alt6_noarch_1445385767.rpm /var/cache/apt/archives/ghc7.6.1_7.6.1-alt6_x86%5f64_1445386600.rpm 
rpm -q ghc7.6.1 -l | xargs filereq haskell.filetrigger.reqs /usr/lib/rpm/haskell.filetrigger
rpm -qf $(< haskell.filetrigger.reqs) | sort -u | xargs rpm -q --qf='%{NAME} '
Comment 7 Ivan Zakharyaschev 2015-12-01 22:11:52 MSK
# while read f; do rpm -qf "$f" --qf="%{NAME} $f\n"; done < haskell.filetrigger.reqs | fgrep gconv
glibc-gconv-modules /usr/lib64/gconv/gconv-modules.cache

So, a missing cache make it work too much.

A cache is something that can be recreated...

The next thing I wonder about is whether (1) recreating the cache once takes too much time (2) there is no way to save the cache (in hasher), and therefore the needed computation is repeated many times.
Comment 8 Ivan Zakharyaschev 2015-12-01 22:40:21 MSK
The cache doesn't seem to be the important thing, but I'm not sure yet.

This might be an inefficiency of a Haskell conversion routine, or this might be an inefficiency of a glibc's API, which shows up if the gconv-modules are absent.

Probably, a simpler test case would be nice.

Something to start with: "ghc-pkg recache" must show the problem (with vs without glibc-gconv-modules).

Unfortunately, I couldn't run the following (hsh-install can't remove glibc-gconv-modules):

[imz@basalt ~]$ hsh --apt-config=/home/imz/.hasher/sisyphus/apt.conf --init  --without-stuff --pkg-init-list=+altlinux-release-sisyphus --pkg-build-list=altlinux-release-sisyphus,basesystem && hsh-install ghc7.6.1- ghc7.6.1  glibc-gconv-modules strace    && hsh-install ghc7.6.1 && hsh-run --rooter ghc-pkg recache && echo OK1 && hsh-install glibc-gconv-modules- && hsh-run --rooter ghc-pkg recache && echo OK2
<86>Dec  1 19:35:32 userdel[10527]: delete user 'rooter'
<86>Dec  1 19:35:32 userdel[10527]: removed group 'rooter' owned by 'rooter'
<86>Dec  1 19:35:32 groupadd[10532]: group added to /etc/group: name=rooter, GID=562
<86>Dec  1 19:35:32 groupadd[10532]: group added to /etc/gshadow: name=rooter
<86>Dec  1 19:35:32 groupadd[10532]: new group: name=rooter, GID=562
<86>Dec  1 19:35:32 useradd[10536]: new user: name=rooter, UID=562, GID=562, home=/root, shell=/bin/bash
<86>Dec  1 19:35:32 userdel[10542]: delete user 'builder'
<86>Dec  1 19:35:32 groupadd[10547]: group added to /etc/group: name=builder, GID=563
<86>Dec  1 19:35:32 groupadd[10547]: group added to /etc/gshadow: name=builder
<86>Dec  1 19:35:32 groupadd[10547]: new group: name=builder, GID=563
<86>Dec  1 19:35:32 useradd[10551]: new user: name=builder, UID=563, GID=563, home=/usr/src, shell=/bin/bash
<13>Dec  1 19:35:33 rpmi: libffi6-1:3.1-alt2 1409163098 installed
<13>Dec  1 19:35:33 rpmi: rpm-macros-ghc7.6.1-common-7.6.1-alt6 1445385767 installed
<13>Dec  1 19:35:33 rpmi: rpm-build-haskell-1-alt26 1445536638 installed
<13>Dec  1 19:35:33 rpmi: haskell-filetrigger-0.0.5-alt2 1445446241 installed
<13>Dec  1 19:35:33 rpmi: libgmp-devel-6.0.0a-alt1 1431516850 installed
<13>Dec  1 19:35:33 rpmi: glib2-locales-2.46.2-alt1 1446853826 installed
<13>Dec  1 19:35:33 rpmi: glib2-2.46.2-alt1 1446853786 installed
<13>Dec  1 19:35:34 rpmi: pkg-config-0.25-alt2 1298849081 installed
<13>Dec  1 19:35:34 rpmi: libffi-devel-1:3.1-alt2 1409163098 installed
<13>Dec  1 19:35:34 rpmi: glibc-gconv-modules-6:2.17-alt8 1389812223 installed
<13>Dec  1 19:35:34 rpmi: strace-4.10.0.484.bb74-alt1 1448507985 installed
<13>Dec  1 19:35:40 rpmi: ghc7.6.1-7.6.1-alt6 1445386600 installed
<13>Dec  1 19:35:40 rpmi: ghc7.6.1-common-7.6.1-alt6 1445385767 installed
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/unix-2.6.0.0/unix.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/unix-2.6.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/time-1.4.0.1/time.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/time-1.4.0.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/template-haskell-2.8.0.0/template-haskell.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/template-haskell-2.8.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/process-1.1.0.2/process.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/process-1.1.0.2 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/pretty-1.1.1.0/pretty.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/pretty-1.1.1.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/old-time-1.1.0.1/old-time.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/old-time-1.1.0.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/old-locale-1.0.0.5/old-locale.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/old-locale-1.0.0.5 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/integer-gmp-0.5.0.0/integer-gmp.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/integer-gmp-0.5.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/hpc-0.6.0.0/hpc.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/hpc-0.6.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/hoopl-3.9.0.0/hoopl.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/hoopl-3.9.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/haskell98-2.0.0.2/haskell98.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/haskell98-2.0.0.2 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/haskell2010-1.1.1.0/haskell2010.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/haskell2010-1.1.1.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/ghc-prim-0.3.0.0/ghc-prim.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/ghc-prim-0.3.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/ghc-7.6.1/ghc.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/ghc-7.6.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/filepath-1.3.0.1/filepath.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/filepath-1.3.0.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/directory-1.2.0.0/directory.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/directory-1.2.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/deepseq-1.3.0.1/deepseq.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/deepseq-1.3.0.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/containers-0.5.0.0/containers.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/containers-0.5.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/bytestring-0.10.0.0/bytestring.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/bytestring-0.10.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/binary-0.5.1.1/binary.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/binary-0.5.1.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/bin-package-db-0.0.0.0/bin-package-db.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/bin-package-db-0.0.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/base-4.6.0.0/base.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/base-4.6.0.0 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/array-0.4.0.1/array.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/array-0.4.0.1 doesn't exist or isn't a directory
Warning: haddock-interfaces: /usr/share/doc/ghc7.6.1-7.6.1/libraries/Cabal-1.16.0/Cabal.haddock doesn't exist or isn't a file
Warning: haddock-html: /usr/share/doc/ghc7.6.1-7.6.1/libraries/Cabal-1.16.0 doesn't exist or isn't a directory
OK1
Чтение списков пакетов...
Построение дерева зависимостей...
Следующие пакеты будут УДАЛЕНЫ:
  glibc-gconv-modules
0 будет обновлено, 0 новых установлено, 1 пакетов будет удалено и 0 не будет обновлено.
Необходимо получить 0B архивов.
После распаковки будет освобождено 6110kB дискового пространства.
Запуск RPM (hsh-rpmi-print-files -e -r /tmp/.private/imz/hasher/aptbox --nodeps)...
hsh-rpmi-print-files: cannot erase packages: glibc-gconv-modules
E: Подпроцесс hsh-rpmi-print-files завершился с ошибкой (1).
hsh-install: Failed to calculate package file list.
hsh-install: Failed to generate package file list.
[imz@basalt ~]$
Comment 9 Ivan Zakharyaschev 2015-12-01 22:49:51 MSK
Here is something to start with. "ghc-pkg recache" could be minimized to get a small test.

hsh --apt-config=/home/imz/.hasher/sisyphus/apt.conf --init  --without-stuff --pkg-init-list=+altlinux-release-sisyphus --pkg-build-list=altlinux-release-sisyphus,basesystem \
&& hsh-install ghc7.6.1- ghc7.6.1  glibc-gconv-modules strace   \
&& hsh-install ghc7.6.1 \
&& echo Recaching1 \
&& hsh-run --rooter ghc-pkg recache \
&& echo OK1 \
&& hsh-run --rooter -- rpm -ev glibc-gconv-modules \
&& echo Recaching2 \
&& hsh-run --rooter ghc-pkg recache \
&& echo OK2

Unfortunately, doing hsh-install glibc-gconv-modules a second time seems not to work.

Now the test passes Recaching1 fast, but Recaching2 takes an eternity.
Comment 10 Repository Robot 2015-12-02 04:14:29 MSK
ghc7.6.1-7.6.1-alt6.1 -> sisyphus:

* Wed Oct 21 2015 Ivan Zakharyaschev <imz@altlinux> 7.6.1-alt6.1
- "ghc-pkg recache" went crazy without glibc-gconv-modules (ALT#31576);
  we workaround it for now without an investigation whether it's a Haskell
  lib or glibc which goes crazy.
- Rearranged Reqs/Provs to make them more clear and to prepare for
  independent co-existence of several versions; not yet ready to make
  ghc7.6.1-common optional (it should be installed only to select the
  default GHC).