35843 2019-01-03 12:51:19 +0300 crashes on boot 2019-01-04 03:08:03 +0300 1 1 4 Development Sisyphus udev unstable all Linux CLOSED FIXED P3 critical --- 1 iv shaba arseny shaba qa-sisyphus oldest_to_newest 177138 0 7931 iv 2019-01-03 12:51:19 +0300 Created attachment 7931 вывод gdb, включая stacktrace После обновления всего systemd до 240 система перестала нормально загружаться. Как оказалось, падает udev. Попробую отакатиться на 239, а пока прикладываю stacktrace, полученный из корки (systemd-coredump внезапно пригодился). 177139 1 iv 2019-01-03 13:17:38 +0300 > Попробую отакатиться на 239 Предсказуемо помогло. 177140 2 iv 2019-01-03 13:25:51 +0300 Ещё немного деталей, вытащенных из корки: backtrace: #0 __strrchr_avx2 () at ../sysdeps/x86_64/multiarch/strrchr-avx2.S:54 #1 0x0000559068f38f81 in device_set_sysname (device=0x55906ab4bd90) at ../src/libsystemd/sd-device/sd-device.c:1005 #2 0x0000559068f3af95 in sd_device_get_sysname (device=device@entry=0x55906ab4bd90, ret=ret@entry=0x7ffc16583390) at ../src/libsystemd/sd-device/sd-device.c:1049 #3 0x0000559068f37b66 in device_append (device=device@entry=0x55906ab4bd90, key=<optimized out>, key@entry=0x7ffc165835f5 "ACTION", _major=_major@entry=0x7ffc16583418, _minor=_minor@entry=0x7ffc16583420, _seqnum=_seqnum@entry=0x7ffc16583428, _action=_action@entry=0x7ffc1658340c) at ../src/libsystemd/sd-device/device-private.c:333 #4 0x0000559068f37cc4 in device_new_from_nulstr (ret=ret@entry=0x7ffc16583490, nulstr=nulstr@entry=0x7ffc165835f5 "ACTION", len=len@entry=162) at ../src/libsystemd/sd-device/device-private.c:450 #5 0x0000559068f3536d in device_monitor_receive_device (m=m@entry=0x55906ab50be0, ret=ret@entry=0x7ffc16585610) at ../src/libsystemd/sd-device/device-monitor.c:453 #6 0x0000559068f35753 in device_monitor_event_handler (s=<optimized out>, fd=<optimized out>, revents=<optimized out>, userdata=0x55906ab50be0) at ../src/libsystemd/sd-device/device-monitor.c:200 #7 0x0000559068f45e90 in source_dispatch (s=s@entry=0x55906ab28e40) at ../src/libsystemd/sd-event/sd-event.c:2816 #8 0x0000559068f4753f in sd_event_dispatch (e=e@entry=0x55906ab35680) at ../src/libsystemd/sd-event/sd-event.c:3229 #9 0x0000559068f476f0 in sd_event_run (e=e@entry=0x55906ab35680, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:3286 #10 0x0000559068f4790b in sd_event_loop (e=0x55906ab35680) at ../src/libsystemd/sd-event/sd-event.c:3308 #11 0x0000559068f0ec6e in main_loop (cgroup=<optimized out>, fd_uevent=3, fd_ctrl=<optimized out>) at ../src/udev/udevd.c:1739 #12 run (argv=<optimized out>, argc=<optimized out>) at ../src/udev/udevd.c:1861 #13 main (argc=<optimized out>, argv=<optimized out>) at ../src/udev/udevd.c:1864 #4 0x0000559068f37cc4 in device_new_from_nulstr (ret=ret@entry=0x7ffc16583490, nulstr=nulstr@entry=0x7ffc165835f5 "ACTION", len=len@entry=162) at ../src/libsystemd/sd-device/device-private.c:450 450 r = device_append(device, key, &major, &minor, &seqnum, &action); #1 0x0000559068f38f81 in device_set_sysname (device=0x55906ab4bd90) at ../src/libsystemd/sd-device/sd-device.c:1005 frame 1: 1005 pos = strrchr(device->devpath, '/'); 1: *device = {n_ref = 1, watch_handle = -1, parent = 0x0, properties = 0x0, properties_iterator = {idx = 0, next_key = 0x0}, properties_generation = 0, properties_iterator_generation = 0, properties_db = 0x0, sysattr_values = 0x0, sysattrs = 0x0, sysattrs_iterator = {idx = 0, next_key = 0x0}, tags = 0x0, tags_iterator = {idx = 0, next_key = 0x0}, tags_generation = 0, tags_iterator_generation = 0, devlinks = 0x0, devlinks_iterator = {idx = 0, next_key = 0x0}, devlinks_generation = 0, devlinks_iterator_generation = 0, devlink_priority = 0, ifindex = 0, devtype = 0x0, devname = 0x0, devnum = 0, properties_strv = 0x0, properties_nulstr = 0x0, properties_nulstr_len = 0, syspath = 0x0, devpath = 0x0, sysnum = 0x0, sysname = 0x0, subsystem = 0x0, driver_subsystem = 0x0, driver = 0x0, id_filename = 0x0, usec_initialized = 0, devmode = 4294967295, devuid = 4294967295, devgid = 4294967295, parent_set = false, sysattrs_read = false, property_tags_outdated = false, property_devlinks_outdated = false, properties_buf_outdated = false, sysname_set = false, subsystem_set = false, driver_subsystem_set = false, driver_set = false, uevent_loaded = false, db_loaded = false, is_initialized = false, sealed = false, db_persist = false} 2: device->devpath = 0x0 strrchr на NULL. Отлично. frame 4: #4 0x0000559068f37cc4 in device_new_from_nulstr (ret=ret@entry=0x7ffc16583490, nulstr=nulstr@entry=0x7ffc165835f5 "ACTION", len=len@entry=162) at ../src/libsystemd/sd-device/device-private.c:450 450 r = device_append(device, key, &major, &minor, &seqnum, &action); 4: key = 0x7ffc165835f5 "ACTION" Куски nullstr (чтобы понять, что добавлялось): 5: nulstr = (uint8_t *) 0x7ffc165835f5 "ACTION" 6: nulstr + 7 = (uint8_t *) 0x7ffc165835fc "bind" 8: nulstr + 12 = (uint8_t *) 0x7ffc16583601 "DEVPATH=/devices/pci0000:00/0000:00:1c.4/0000:3b:00.0/mdio_bus/r8169-3b00/r8169-3b00:00" 12: nulstr + 100 = (uint8_t *) 0x7ffc16583659 "SUBSYSTEM=mdio_bus" 18: nulstr + 119 = (uint8_t *) 0x7ffc1658366c "DEVTYPE=PHY" 23: nulstr + 131 = (uint8_t *) 0x7ffc16583678 "DRIVER=Generic PHY" 27: nulstr + 150 = (uint8_t *) 0x7ffc1658368b "SEQNUM=3181" Похоже, добавлялся $ lspci | grep ^3b 3b:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15) 177141 3 iv 2019-01-03 14:57:31 +0300 Сходил к апстриму: https://github.com/systemd/systemd/issues/11319 Там расписал, что происходит. Попробую починить. 177143 4 iv 2019-01-03 15:52:29 +0300 Нашёлся патчик, который должен помочь: https://github.com/systemd/systemd/commit/18fee12a2d489378a2a9b647db0d0eb8c43f5362 Прошу приложить. Очень похожее изменение работает у меня прямо сейчас. 177147 5 shaba 2019-01-04 03:08:03 +0300 fixed in systemd-240-alt2. 7931 2019-01-03 12:51:19 +0300 2019-01-03 12:51:19 +0300 вывод gdb, включая stacktrace udevd-debug.txt text/plain 4855 iv R05VIGdkYiAoR0RCKSA4LjIuNTAuMjAxODA5MTctYWx0MiAoQUxUIFNpc3lwaHVzKQpDb3B5cmln aHQgKEMpIDIwMTggRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uLCBJbmMuCkxpY2Vuc2UgR1BMdjMr OiBHTlUgR1BMIHZlcnNpb24gMyBvciBsYXRlciA8aHR0cDovL2dudS5vcmcvbGljZW5zZXMvZ3Bs Lmh0bWw+ClRoaXMgaXMgZnJlZSBzb2Z0d2FyZTogeW91IGFyZSBmcmVlIHRvIGNoYW5nZSBhbmQg cmVkaXN0cmlidXRlIGl0LgpUaGVyZSBpcyBOTyBXQVJSQU5UWSwgdG8gdGhlIGV4dGVudCBwZXJt aXR0ZWQgYnkgbGF3LgpUeXBlICJzaG93IGNvcHlpbmciIGFuZCAic2hvdyB3YXJyYW50eSIgZm9y IGRldGFpbHMuClRoaXMgR0RCIHdhcyBjb25maWd1cmVkIGFzICJ4ODZfNjQtYWx0LWxpbnV4Ii4K VHlwZSAic2hvdyBjb25maWd1cmF0aW9uIiBmb3IgY29uZmlndXJhdGlvbiBkZXRhaWxzLgpGb3Ig YnVnIHJlcG9ydGluZyBpbnN0cnVjdGlvbnMsIHBsZWFzZSBzZWU6CjxodHRwOi8vd3d3LmdudS5v cmcvc29mdHdhcmUvZ2RiL2J1Z3MvPi4KRmluZCB0aGUgR0RCIG1hbnVhbCBhbmQgb3RoZXIgZG9j dW1lbnRhdGlvbiByZXNvdXJjZXMgb25saW5lIGF0OgogICAgPGh0dHA6Ly93d3cuZ251Lm9yZy9z b2Z0d2FyZS9nZGIvZG9jdW1lbnRhdGlvbi8+LgoKRm9yIGhlbHAsIHR5cGUgImhlbHAiLgpUeXBl ICJhcHJvcG9zIHdvcmQiIHRvIHNlYXJjaCBmb3IgY29tbWFuZHMgcmVsYXRlZCB0byAid29yZCIu Li4KUmVhZGluZyBzeW1ib2xzIGZyb20gL2xpYi9zeXN0ZW1kL3N5c3RlbWQtdWRldmQuLi4KUmVh ZGluZyBzeW1ib2xzIGZyb20gL3Vzci9saWIvZGVidWcvbGliL3N5c3RlbWQvc3lzdGVtZC11ZGV2 ZC5kZWJ1Zy4uLgpbTmV3IExXUCAyMjE4XQpbVGhyZWFkIGRlYnVnZ2luZyB1c2luZyBsaWJ0aHJl YWRfZGIgZW5hYmxlZF0KVXNpbmcgaG9zdCBsaWJ0aHJlYWRfZGIgbGlicmFyeSAiL2xpYjY0L2xp YnRocmVhZF9kYi5zby4xIi4KQ29yZSB3YXMgZ2VuZXJhdGVkIGJ5IGAvbGliL3N5c3RlbWQvc3lz dGVtZC11ZGV2ZCcuClByb2dyYW0gdGVybWluYXRlZCB3aXRoIHNpZ25hbCBTSUdTRUdWLCBTZWdt ZW50YXRpb24gZmF1bHQuCiMwICBfX3N0cnJjaHJfYXZ4MiAoKSBhdCAuLi9zeXNkZXBzL3g4Nl82 NC9tdWx0aWFyY2gvc3RycmNoci1hdngyLlM6NTQKNTQJCXZtb3ZkcXUJKCVyZGkpLCAleW1tMQoo Z2RiKSBidAojMCAgX19zdHJyY2hyX2F2eDIgKCkgYXQgLi4vc3lzZGVwcy94ODZfNjQvbXVsdGlh cmNoL3N0cnJjaHItYXZ4Mi5TOjU0CiMxICAweDAwMDA1NTkwNjhmMzhmODEgaW4gZGV2aWNlX3Nl dF9zeXNuYW1lIChkZXZpY2U9MHg1NTkwNmFiNGJkOTApIGF0IC4uL3NyYy9saWJzeXN0ZW1kL3Nk LWRldmljZS9zZC1kZXZpY2UuYzoxMDA1CiMyICAweDAwMDA1NTkwNjhmM2FmOTUgaW4gc2RfZGV2 aWNlX2dldF9zeXNuYW1lIChkZXZpY2U9ZGV2aWNlQGVudHJ5PTB4NTU5MDZhYjRiZDkwLCByZXQ9 cmV0QGVudHJ5PTB4N2ZmYzE2NTgzMzkwKSBhdCAuLi9zcmMvbGlic3lzdGVtZC9zZC1kZXZpY2Uv c2QtZGV2aWNlLmM6MTA0OQojMyAgMHgwMDAwNTU5MDY4ZjM3YjY2IGluIGRldmljZV9hcHBlbmQg KGRldmljZT1kZXZpY2VAZW50cnk9MHg1NTkwNmFiNGJkOTAsIGtleT08b3B0aW1pemVkIG91dD4s IGtleUBlbnRyeT0weDdmZmMxNjU4MzVmNSAiQUNUSU9OIiwgX21ham9yPV9tYWpvckBlbnRyeT0w eDdmZmMxNjU4MzQxOCwgCiAgICBfbWlub3I9X21pbm9yQGVudHJ5PTB4N2ZmYzE2NTgzNDIwLCBf c2VxbnVtPV9zZXFudW1AZW50cnk9MHg3ZmZjMTY1ODM0MjgsIF9hY3Rpb249X2FjdGlvbkBlbnRy eT0weDdmZmMxNjU4MzQwYykgYXQgLi4vc3JjL2xpYnN5c3RlbWQvc2QtZGV2aWNlL2RldmljZS1w cml2YXRlLmM6MzMzCiM0ICAweDAwMDA1NTkwNjhmMzdjYzQgaW4gZGV2aWNlX25ld19mcm9tX251 bHN0ciAocmV0PXJldEBlbnRyeT0weDdmZmMxNjU4MzQ5MCwgbnVsc3RyPW51bHN0ckBlbnRyeT0w eDdmZmMxNjU4MzVmNSAiQUNUSU9OIiwgbGVuPWxlbkBlbnRyeT0xNjIpCiAgICBhdCAuLi9zcmMv bGlic3lzdGVtZC9zZC1kZXZpY2UvZGV2aWNlLXByaXZhdGUuYzo0NTAKIzUgIDB4MDAwMDU1OTA2 OGYzNTM2ZCBpbiBkZXZpY2VfbW9uaXRvcl9yZWNlaXZlX2RldmljZSAobT1tQGVudHJ5PTB4NTU5 MDZhYjUwYmUwLCByZXQ9cmV0QGVudHJ5PTB4N2ZmYzE2NTg1NjEwKSBhdCAuLi9zcmMvbGlic3lz dGVtZC9zZC1kZXZpY2UvZGV2aWNlLW1vbml0b3IuYzo0NTMKIzYgIDB4MDAwMDU1OTA2OGYzNTc1 MyBpbiBkZXZpY2VfbW9uaXRvcl9ldmVudF9oYW5kbGVyIChzPTxvcHRpbWl6ZWQgb3V0PiwgZmQ9 PG9wdGltaXplZCBvdXQ+LCByZXZlbnRzPTxvcHRpbWl6ZWQgb3V0PiwgdXNlcmRhdGE9MHg1NTkw NmFiNTBiZTApCiAgICBhdCAuLi9zcmMvbGlic3lzdGVtZC9zZC1kZXZpY2UvZGV2aWNlLW1vbml0 b3IuYzoyMDAKIzcgIDB4MDAwMDU1OTA2OGY0NWU5MCBpbiBzb3VyY2VfZGlzcGF0Y2ggKHM9c0Bl bnRyeT0weDU1OTA2YWIyOGU0MCkgYXQgLi4vc3JjL2xpYnN5c3RlbWQvc2QtZXZlbnQvc2QtZXZl bnQuYzoyODE2CiM4ICAweDAwMDA1NTkwNjhmNDc1M2YgaW4gc2RfZXZlbnRfZGlzcGF0Y2ggKGU9 ZUBlbnRyeT0weDU1OTA2YWIzNTY4MCkgYXQgLi4vc3JjL2xpYnN5c3RlbWQvc2QtZXZlbnQvc2Qt ZXZlbnQuYzozMjI5CiM5ICAweDAwMDA1NTkwNjhmNDc2ZjAgaW4gc2RfZXZlbnRfcnVuIChlPWVA ZW50cnk9MHg1NTkwNmFiMzU2ODAsIHRpbWVvdXQ9dGltZW91dEBlbnRyeT0xODQ0Njc0NDA3Mzcw OTU1MTYxNSkgYXQgLi4vc3JjL2xpYnN5c3RlbWQvc2QtZXZlbnQvc2QtZXZlbnQuYzozMjg2CiMx MCAweDAwMDA1NTkwNjhmNDc5MGIgaW4gc2RfZXZlbnRfbG9vcCAoZT0weDU1OTA2YWIzNTY4MCkg YXQgLi4vc3JjL2xpYnN5c3RlbWQvc2QtZXZlbnQvc2QtZXZlbnQuYzozMzA4CiMxMSAweDAwMDA1 NTkwNjhmMGVjNmUgaW4gbWFpbl9sb29wIChjZ3JvdXA9PG9wdGltaXplZCBvdXQ+LCBmZF91ZXZl bnQ9MywgZmRfY3RybD08b3B0aW1pemVkIG91dD4pIGF0IC4uL3NyYy91ZGV2L3VkZXZkLmM6MTcz OQojMTIgcnVuIChhcmd2PTxvcHRpbWl6ZWQgb3V0PiwgYXJnYz08b3B0aW1pemVkIG91dD4pIGF0 IC4uL3NyYy91ZGV2L3VkZXZkLmM6MTg2MQojMTMgbWFpbiAoYXJnYz08b3B0aW1pemVkIG91dD4s IGFyZ3Y9PG9wdGltaXplZCBvdXQ+KSBhdCAuLi9zcmMvdWRldi91ZGV2ZC5jOjE4NjQKKGdkYikg cXVpdAogICAgICAgICAgIFBJRDogMjIxOCAoc3lzdGVtZC11ZGV2ZCkKICAgICAgICAgICBVSUQ6 IDAgKHJvb3QpCiAgICAgICAgICAgR0lEOiAwIChyb290KQogICAgICAgIFNpZ25hbDogMTEgKFNF R1YpCiAgICAgVGltZXN0YW1wOiBUaHUgMjAxOS0wMS0wMyAxMzoyOTowOSArMDQgKDEybWluIGFn bykKICBDb21tYW5kIExpbmU6IC9saWIvc3lzdGVtZC9zeXN0ZW1kLXVkZXZkCiAgICBFeGVjdXRh YmxlOiAvbGliL3N5c3RlbWQvc3lzdGVtZC11ZGV2ZAogQ29udHJvbCBHcm91cDogL3N5c3RlbS5z bGljZS9zeXN0ZW1kLXVkZXZkLnNlcnZpY2UKICAgICAgICAgIFVuaXQ6IHN5c3RlbWQtdWRldmQu c2VydmljZQogICAgICAgICBTbGljZTogc3lzdGVtLnNsaWNlCiAgICAgICBCb290IElEOiA5YmI0 YmFjYmQ5ZTM0M2U2YTMyOWZlNzE1Nzg1YTFmOAogICAgTWFjaGluZSBJRDogYTgyYzg3NzFkYTVi OTg5NjIxNDcyMDQyNWIxNjgwYjQKICAgICAgSG9zdG5hbWU6IHRpdGFuLmxvY2FsZG9tYWluCiAg ICAgICBTdG9yYWdlOiAvdmFyL2xpYi9zeXN0ZW1kL2NvcmVkdW1wL2NvcmUuc3lzdGVtZC11ZGV2 ZC4wLjliYjRiYWNiZDllMzQzZTZhMzI5ZmU3MTU3ODVhMWY4LjIyMTguMTU0NjUwNzc0OTAwMDAw MC5sejQKICAgICAgIE1lc3NhZ2U6IFByb2Nlc3MgMjIxOCAoc3lzdGVtZC11ZGV2ZCkgb2YgdXNl ciAwIGR1bXBlZCBjb3JlLgogICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICBTdGFjayB0 cmFjZSBvZiB0aHJlYWQgMjIxODoKICAgICAgICAgICAgICAgICMwICAweDAwMDA3ZjMwM2U4N2Yw ZDcgX19zdHJyY2hyX2F2eDIgKGxpYmMuc28uNikKICAgICAgICAgICAgICAgICMxICAweDAwMDA1 NTkwNjhmMzhmODEgbi9hIChzeXN0ZW1kLXVkZXZkKQogICAgICAgICAgICAgICAgIzIgIDB4MDAw MDU1OTA2OGYzYWY5NSBuL2EgKHN5c3RlbWQtdWRldmQpCiAgICAgICAgICAgICAgICAjMyAgMHgw MDAwNTU5MDY4ZjM3YjY2IG4vYSAoc3lzdGVtZC11ZGV2ZCkKICAgICAgICAgICAgICAgICM0ICAw eDAwMDA1NTkwNjhmMzdjYzQgbi9hIChzeXN0ZW1kLXVkZXZkKQogICAgICAgICAgICAgICAgIzUg IDB4MDAwMDU1OTA2OGYzNTM2ZCBuL2EgKHN5c3RlbWQtdWRldmQpCiAgICAgICAgICAgICAgICAj NiAgMHgwMDAwNTU5MDY4ZjM1NzUzIG4vYSAoc3lzdGVtZC11ZGV2ZCkKICAgICAgICAgICAgICAg ICM3ICAweDAwMDA1NTkwNjhmNDVlOTAgbi9hIChzeXN0ZW1kLXVkZXZkKQogICAgICAgICAgICAg ICAgIzggIDB4MDAwMDU1OTA2OGY0NzUzZiBuL2EgKHN5c3RlbWQtdWRldmQpCiAgICAgICAgICAg ICAgICAjOSAgMHgwMDAwNTU5MDY4ZjQ3NmYwIG4vYSAoc3lzdGVtZC11ZGV2ZCkKICAgICAgICAg ICAgICAgICMxMCAweDAwMDA1NTkwNjhmNDc5MGIgbi9hIChzeXN0ZW1kLXVkZXZkKQogICAgICAg ICAgICAgICAgIzExIDB4MDAwMDU1OTA2OGYwZWM2ZSBuL2EgKHN5c3RlbWQtdWRldmQpCiAgICAg ICAgICAgICAgICAjMTIgMHgwMDAwN2YzMDNlNzQ4MDhiIF9fbGliY19zdGFydF9tYWluIChsaWJj LnNvLjYpCiAgICAgICAgICAgICAgICAjMTMgMHgwMDAwNTU5MDY4ZjBmM2NhIG4vYSAoc3lzdGVt ZC11ZGV2ZCkKCg==