Attachment 18281 Details for Bug 53963 – crio.log

crio.log

crio.log (text/x-log), 17.36 KB, created by Artem Varaksa on 2025-04-23 18:40:01 MSK

(hide)

Description:

Filename:

MIME Type:

Creator: Artem Varaksa

Created: 2025-04-23 18:40:01 MSK

Size: 17.36 KB

patch

obsolete

>Ð°Ð¿Ñ 23 18:08:59 kubehosta systemd[1]: Stopping crio.service - Container Runtime Interface for OCI (CRI-O)...
>Ð°Ð¿Ñ 23 18:09:00 kubehosta systemd[1]: crio.service: Deactivated successfully.
>Ð°Ð¿Ñ 23 18:09:00 kubehosta systemd[1]: Stopped crio.service - Container Runtime Interface for OCI (CRI-O).
>Ð°Ð¿Ñ 23 18:09:00 kubehosta systemd[1]: crio.service: Consumed 8.451s CPU time, 385.5M memory peak.
>Ð°Ð¿Ñ 23 18:09:09 kubehosta systemd[1]: Starting crio.service - Container Runtime Interface for OCI (CRI-O)...
>Ð°Ð¿Ñ 23 18:09:09 kubehosta (crio)[7471]: crio.service: Referenced but unset environment variable evaluates to an empty string: CRIO_CONFIG_OPTIONS, CRIO_METRICS_OPTIONS, CRIO_NETWORK_OPTIONS, CRIO_RUNTIME_OPTIONS, CRIO_STORAGE_OPTIONS
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.476567906+03:00" level=info msg="Updating config from single file: /etc/crio/crio.conf"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.476596507+03:00" level=info msg="Updating config from drop-in file: /etc/crio/crio.conf"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.477526197+03:00" level=info msg="Updating config from path: /etc/crio/crio.conf.d"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.487755868+03:00" level=info msg="Starting CRI-O, version: 1.31.7, git: unknown(clean)"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.487816725+03:00" level=info msg="Node configuration value for hugetlb cgroup is true"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.487824864+03:00" level=info msg="Node configuration value for pid cgroup is true"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.487866364+03:00" level=info msg="Node configuration value for memoryswap cgroup is true"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.487873538+03:00" level=info msg="Node configuration value for cgroup v2 is true"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.494134475+03:00" level=info msg="Node configuration value for systemd AllowedCPUs is true"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.494428769+03:00" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.495104980+03:00" level=info msg="Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.497395897+03:00" level=info msg="Using runtime handler crun version 1.21, commit: 000fa0d4eeed8938301f3bcf8206405315bc1017, rundir: /run/crun, spec: 1.0.0, +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.501706024+03:00" level=info msg="Checkpoint/restore support disabled: CRIU binary not found int $PATH"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.501722647+03:00" level=info msg="Using the internal default seccomp profile"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.501729747+03:00" level=info msg="AppArmor is disabled by the system or at CRI-O build-time"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.501736222+03:00" level=info msg="No blockio config file specified, blockio not configured"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.501742500+03:00" level=info msg="RDT not available in the host system"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.501753037+03:00" level=info msg="Using conmon executable: /usr/bin/conmon"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.503151950+03:00" level=info msg="Conmon does support the --sync option"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.503165988+03:00" level=info msg="Conmon does support the --log-global-size-max option"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.507228933+03:00" level=info msg="Found CNI network cbr0 (type=flannel) at /etc/cni/net.d/10-flannel.conflist"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.507280756+03:00" level=error msg="Error loading CNI config list file /etc/cni/net.d/100-crio-bridge.conflist: error parsing configuration list: invalid character '#' looking for beginning of value"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.507290838+03:00" level=info msg="Updated default CNI network name to cbr0"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.507718367+03:00" level=info msg="Current CRI-O configuration:\n[crio]\n  root = \"/var/lib/containers/storage\"\n  runroot = \"/var/run/containers/storage\"\n  imagestore = \"\"\n  storage_driver = \"overlay\"\n  storage_option = [\"overlay.mountopt=nodev,metacopy=on\", \"overlay.override_kernel_check=1\"]\n  log_dir = \"/var/log/crio/pods\"\n  version_file = \"/var/run/crio/version\"\n  version_file_persist = \"\"\n  clean_shutdown_file = \"/var/lib/crio/clean.shutdown\"\n  internal_wipe = true\n  internal_repair = true\n  [crio.api]\n    grpc_max_send_msg_size = 83886080\n    grpc_max_recv_msg_size = 83886080\n    listen = \"/var/run/crio/crio.sock\"\n    stream_address = \"127.0.0.1\"\n    stream_port = \"0\"\n    stream_enable_tls = false\n    stream_tls_cert = \"\"\n    stream_tls_key = \"\"\n    stream_tls_ca = \"\"\n    stream_idle_timeout = \"\"\n  [crio.runtime]\n    no_pivot = false\n    selinux = false\n    log_to_journald = false\n    drop_infra_ctr = true\n    read_only = false\n    hooks_dir = [\"/usr/share/containers/oci/hooks.d\"]\n    default_capabilities = [\"CHOWN\", \"DAC_OVERRIDE\", \"FSETID\", \"FOWNER\", \"SETGID\", \"SETUID\", \"SETPCAP\", \"NET_BIND_SERVICE\", \"KILL\"]\n    add_inheritable_capabilities = false\n    allowed_devices = [\"/dev/fuse\", \"/dev/net/tun\"]\n    cdi_spec_dirs = [\"/etc/cdi\", \"/var/run/cdi\"]\n    device_ownership_from_security_context = false\n    default_runtime = \"crun\"\n    decryption_keys_path = \"/etc/crio/keys/\"\n    conmon = \"/usr/bin/conmon\"\n    conmon_cgroup = \"\"\n    seccomp_profile = \"\"\n    apparmor_profile = \"crio-default\"\n    blockio_config_file = \"\"\n    blockio_reload = false\n    irqbalance_config_file = \"/etc/sysconfig/irqbalance\"\n    rdt_config_file = \"\"\n    cgroup_manager = \"systemd\"\n    default_mounts_file = \"\"\n    container_exits_dir = \"/var/run/crio/exits\"\n    container_attach_socket_dir = \"/var/run/crio\"\n    bind_mount_prefix = \"\"\n    uid_mappings = \"\"\n    minimum_mappable_uid = -1\n    gid_mappings = \"\"\n    minimum_mappable_gid = -1\n    log_level = \"info\"\n    log_filter = \"\"\n    namespaces_dir = \"/var/run\"\n    pinns_path = \"/usr/bin/pinns\"\n    enable_criu_support = false\n    pids_limit = -1\n    log_size_max = -1\n    ctr_stop_timeout = 30\n    separate_pull_cgroup = \"\"\n    infra_ctr_cpuset = \"\"\n    shared_cpuset = \"\"\n    enable_pod_events = false\n    irqbalance_config_restore_file = \"/etc/sysconfig/orig_irq_banned_cpus\"\n    hostnetwork_disable_selinux = true\n    disable_hostport_mapping = false\n    timezone = \"\"\n    [crio.runtime.runtimes]\n      [crio.runtime.runtimes.crun]\n        runtime_config_path = \"\"\n        runtime_path = \"/usr/bin/crun\"\n        runtime_type = \"oci\"\n        runtime_root = \"/run/crun\"\n        allowed_annotations = [\"io.containers.trace-syscall\", \"io.kubernetes.cri-o.Devices\"]\n        monitor_path = \"/usr/bin/conmon\"\n        monitor_cgroup = \"system.slice\"\n        monitor_env = [\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"]\n        container_min_memory = \"500KiB\"\n        no_sync_log = false\n  [crio.image]\n    default_transport = \"docker://\"\n    global_auth_file = \"\"\n    pause_image = \"registry.k8s.io/pause:3.10\"\n    pause_image_auth_file = \"\"\n    pause_command = \"/pause\"\n    signature_policy = \"\"\n    signature_policy_dir = \"/etc/crio/policies\"\n    image_volumes = \"mkdir\"\n    big_files_temporary_dir = \"\"\n    auto_reload_registries = false\n    pull_progress_timeout = \"10s\"\n  [crio.network]\n    cni_default_network = \"\"\n    network_dir = \"/etc/cni/net.d/\"\n    plugin_dirs = [\"/usr/libexec/cni\", \"/opt/cni/bin\"]\n  [crio.metrics]\n    enable_metrics = false\n    metrics_collectors = [\"image_pulls_layer_size\", \"containers_events_dropped_total\", \"containers_oom_total\", \"processes_defunct\", \"operations_total\", \"operations_latency_seconds\", \"operations_latency_seconds_total\", \"operations_errors_total\", \"image_pulls_bytes_total\", \"image_pulls_skipped_bytes_total\", \"image_pulls_failure_total\", \"image_pulls_success_total\", \"image_layer_reuse_total\", \"containers_oom_count_total\", \"containers_seccomp_notifier_count_total\", \"resources_stalled_at_stage\"]\n    metrics_host = \"127.0.0.1\"\n    metrics_port = 9090\n    metrics_socket = \"\"\n    metrics_cert = \"\"\n    metrics_key = \"\"\n  [crio.tracing]\n    enable_tracing = false\n    tracing_endpoint = \"127.0.0.1:4317\"\n    tracing_sampling_rate_per_million = 0\n  [crio.stats]\n    stats_collection_period = 0\n    collection_period = 0\n  [crio.nri]\n    enable_nri = true\n    nri_listen = \"/var/run/nri/nri.sock\"\n    nri_plugin_dir = \"/opt/nri/plugins\"\n    nri_plugin_config_dir = \"/etc/nri/conf.d\"\n    nri_plugin_registration_timeout = \"5s\"\n    nri_plugin_request_timeout = \"2s\"\n    nri_disable_connections = false\n"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.508421962+03:00" level=info msg="Attempting to restore irqbalance config from /etc/sysconfig/orig_irq_banned_cpus"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.508504601+03:00" level=info msg="Restore irqbalance config: nothing to do"
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.525651874+03:00" level=error msg="Failed to update container state for 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707: stdout: , stderr: "
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.540968042+03:00" level=error msg="Failed to update container state for ecd44545c684fcdee97c5bd657911044801c993b11103b57c5a99ad658b9f365: stdout: , stderr: "
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.549598121+03:00" level=error msg="Failed to update container state for 944ccec6e90a2b8e40ea5233dafc6320aab6a3b4a1819a8ce833e1f2e957778f: stdout: , stderr: "
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.560416317+03:00" level=error msg="Failed to update container state for 65c3bede7cff7bc65d3652b925e074bb3f7af826a37a9c0d9f87755d31ffaeb5: stdout: , stderr: "
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.569232870+03:00" level=error msg="Failed to update container state for a1ba3945a1a990b798a73cbc46f841fc2cd69e72d5ca0c9fe16aeb003bb74e13: stdout: , stderr: "
>Ð°Ð¿Ñ 23 18:09:09 kubehosta crio[7471]: time="2025-04-23 18:09:09.583897267+03:00" level=error msg="Failed to update container state for 836fcdd9cab88816c1a8881e56658b878d3a24ca6f1835994198c3135a8a50b3: stdout: , stderr: "
>Ð°Ð¿Ñ 23 18:09:19 kubehosta crio[7471]: time="2025-04-23 18:09:19.602838047+03:00" level=warning msg="Stopping container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 with stop signal timed out. Killing..."
>Ð°Ð¿Ñ 23 18:09:19 kubehosta crio[7471]: time="2025-04-23 18:09:19.605332359+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:19 kubehosta crio[7471]: time="2025-04-23 18:09:19.650267874+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:19 kubehosta crio[7471]: time="2025-04-23 18:09:19.712915504+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:19 kubehosta crio[7471]: time="2025-04-23 18:09:19.800722517+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:20 kubehosta crio[7471]: time="2025-04-23 18:09:20.157040173+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:20 kubehosta crio[7471]: time="2025-04-23 18:09:20.612796096+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:21 kubehosta crio[7471]: time="2025-04-23 18:09:21.872214589+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:23 kubehosta crio[7471]: time="2025-04-23 18:09:23.190239388+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:26 kubehosta crio[7471]: time="2025-04-23 18:09:26.300308640+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:32 kubehosta crio[7471]: time="2025-04-23 18:09:32.144051388+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:09:49 kubehosta crio[7471]: time="2025-04-23 18:09:49.340521773+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:10:25 kubehosta crio[7471]: time="2025-04-23 18:10:25.469679979+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"
>Ð°Ð¿Ñ 23 18:11:14 kubehosta crio[7471]: time="2025-04-23 18:11:14.062217531+03:00" level=error msg="Killing container 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 failed: `/usr/bin/crun --root /run/crun --systemd-cgroup kill 452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707 KILL` failed: error opening file `/run/crun/452c4473cc4a97483daa7d4fcbd304ccab8adbdba748c59149929ea98ee41707/status`: No such file or directory\n : exit status 1"

Actions: View

Attachments on bug 53963: 18281