Attachment 11005 Details for Bug 43104 – clickhouse-backup-ftp.log

clickhouse-backup-ftp.log

clickhouse-backup-ftp.log (text/x-log), 9.68 KB, created by Alexander Makeenkov on 2022-06-29 13:48:11 MSK

(hide)

Description:

Filename:

MIME Type:

Creator: Alexander Makeenkov

Created: 2022-06-29 13:48:11 MSK

Size: 9.68 KB

patch

obsolete

># LOG_LEVEL=debug clickhouse-backup create_remote ftp_backup
>2022/06/29 13:46:52.671247  info SELECT name, engine FROM system.databases WHERE name NOT IN ('system', 'INFORMATION_SCHEMA', 'information_schema')
>2022/06/29 13:46:52.675567  info SHOW CREATE DATABASE `default`
>2022/06/29 13:46:52.682208  info SHOW CREATE DATABASE `test_db`
>2022/06/29 13:46:52.684755  info SELECT count() FROM system.settings WHERE name = 'show_table_uuid_in_table_create_query_if_not_nil'
>2022/06/29 13:46:52.691516  info SELECT name FROM system.databases WHERE engine IN ('MySQL','PostgreSQL')
>2022/06/29 13:46:52.696351  info
>                SELECT
>                        countIf(name='data_path') is_data_path_present,
>                        countIf(name='data_paths') is_data_paths_present,
>                        countIf(name='uuid') is_uuid_present,
>                        countIf(name='create_table_query') is_create_table_query_present,
>                        countIf(name='total_bytes') is_total_bytes_present
>                FROM system.columns WHERE database='system' AND table='tables'
>
>2022/06/29 13:46:52.707246  info SELECT database, name, engine , data_paths , uuid , create_table_query , coalesce(total_bytes, 0) AS total_bytes   FROM system.tables WHERE is_temporary = 0 SETTINGS show_table_uuid_in_table_create_query_if_not_nil=1
>2022/06/29 13:46:52.736057  info SELECT value FROM `system`.`build_options` where name='VERSION_INTEGER'
>2022/06/29 13:46:52.741687  info SELECT * FROM system.disks;
>2022/06/29 13:46:52.750971 debug create data               backup=ftp_backup operation=create table=test_db.test_table
>2022/06/29 13:46:52.751163  info ALTER TABLE `test_db`.`test_table` FREEZE WITH NAME 'fad45c962b83470493190e4e1e05e50b';
>2022/06/29 13:46:52.816234 debug freezed                   backup=ftp_backup operation=create table=test_db.test_table
>2022/06/29 13:46:52.821656 debug shadow moved              backup=ftp_backup disk=default operation=create table=test_db.test_table
>2022/06/29 13:46:52.823079 debug done                      backup=ftp_backup operation=create table=test_db.test_table
>2022/06/29 13:46:52.823218 debug create metadata           backup=ftp_backup operation=create table=test_db.test_table
>2022/06/29 13:46:52.824515  info done                      backup=ftp_backup operation=create table=test_db.test_table
>2022/06/29 13:46:52.824669  info SELECT toUInt8(count()) udf_presents FROM system.columns WHERE database='system' AND table='functions' AND name='create_query'
>2022/06/29 13:46:52.839652  info SELECT name, create_query FROM system.functions WHERE create_query!=''
>2022/06/29 13:46:52.851734  info SELECT value FROM `system`.`build_options` where name='VERSION_DESCRIBE'
>2022/06/29 13:46:52.860259  info done                      backup=ftp_backup duration=196ms operation=create
>2022/06/29 13:46:52.871515  info SELECT value FROM `system`.`build_options` where name='VERSION_INTEGER'
>2022/06/29 13:46:52.877605  info SELECT * FROM system.disks;
>2022/06/29 13:46:52.888433  info SELECT max(toInt64(bytes_on_disk * 1.02)) AS max_file_size FROM system.parts
>2022/06/29 13:46:52.899230  info SELECT count() AS is_macros_exists FROM system.tables WHERE database='system' AND table='macros'
>2022/06/29 13:46:52.911006  info SELECT * FROM system.macros
>2022/06/29 13:46:52.918964 debug /tmp/.private/root/.clickhouse-backup-metadata.cache.FTP load 0 elements
>2022/06/29 13:46:52.919022 debug FTP::getConnectionFromPool(Walk) active=0 idle=0
>2022/06/29 13:46:53.266836 debug FTP::returnConnectionToPool(Walk) active=1 idle=0
>2022/06/29 13:46:53.267263 debug /tmp/.private/root/.clickhouse-backup-metadata.cache.FTP save 0 elements
>2022/06/29 13:46:53.268024 debug prepare table concurrent semaphore with concurrency=16 len(tablesForUpload)=1 backup=ftp_backup operation=upload
>2022/06/29 13:46:53.268337 debug start uploadTableData test_db.test_table with concurrency=16 len(table.Parts[...])=4
>2022/06/29 13:46:53.269423 debug start upload 7 files to ftp_backup/shadow/test_db/test_table/default_all_4_4_0.tar
>2022/06/29 13:46:53.269619 debug FTP::getConnectionFromPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_4_4_0.tar) active=0 idle=1
>2022/06/29 13:46:53.269508 debug start upload 7 files to ftp_backup/shadow/test_db/test_table/default_all_3_3_0.tar
>2022/06/29 13:46:53.269914 debug FTP::getConnectionFromPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_3_3_0.tar) active=1 idle=0
>2022/06/29 13:46:53.269518 debug start upload 7 files to ftp_backup/shadow/test_db/test_table/default_all_1_1_0.tar
>2022/06/29 13:46:53.270104 debug FTP::getConnectionFromPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_1_1_0.tar) active=1 idle=0
>2022/06/29 13:46:53.269541 debug start upload 7 files to ftp_backup/shadow/test_db/test_table/default_all_2_2_0.tar
>2022/06/29 13:46:53.270551 debug FTP::getConnectionFromPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_2_2_0.tar) active=1 idle=0
>2022/06/29 13:46:53.272991 debug FTP::returnConnectionToPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_4_4_0.tar) active=1 idle=0
>2022/06/29 13:46:53.273701 debug FTP::PutFile key=ftp_backup/shadow/test_db/test_table/default_all_4_4_0.tar
>2022/06/29 13:46:53.274137 debug FTP::getConnectionFromPool(PutFile) active=0 idle=1
>2022/06/29 13:46:53.275775  warn FTP::MkdirAll MakeDir(ftp_backup) return error: 550 ftp_backup: Permission denied
>2022/06/29 13:46:53.276609  warn FTP::MkdirAll MakeDir(ftp_backup/shadow) return error: 550 ftp_backup/shadow: No such file or directory
>2022/06/29 13:46:53.277800  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db) return error: 550 ftp_backup/shadow/test_db: No such file or directory
>2022/06/29 13:46:53.278655  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db/test_table) return error: 550 ftp_backup/shadow/test_db/test_table: No such file or directory
>2022/06/29 13:46:53.281244 debug FTP::returnConnectionToPool(PutFile) active=1 idle=0
>2022/06/29 13:46:53.281602 error UploadCompressedStream return error: 550 ftp_backup/shadow/test_db/test_table/default_all_4_4_0.tar: No such file or directory
>2022/06/29 13:46:53.347476 debug FTP::returnConnectionToPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_3_3_0.tar) active=1 idle=1
>2022/06/29 13:46:53.347884 debug FTP::PutFile key=ftp_backup/shadow/test_db/test_table/default_all_3_3_0.tar
>2022/06/29 13:46:53.347926 debug FTP::getConnectionFromPool(PutFile) active=0 idle=2
>2022/06/29 13:46:53.349771  warn FTP::MkdirAll MakeDir(ftp_backup) return error: 550 ftp_backup: Permission denied
>2022/06/29 13:46:53.350176  warn FTP::MkdirAll MakeDir(ftp_backup/shadow) return error: 550 ftp_backup/shadow: No such file or directory
>2022/06/29 13:46:53.350577  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db) return error: 550 ftp_backup/shadow/test_db: No such file or directory
>2022/06/29 13:46:53.351024  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db/test_table) return error: 550 ftp_backup/shadow/test_db/test_table: No such file or directory
>2022/06/29 13:46:53.352713 debug FTP::returnConnectionToPool(PutFile) active=1 idle=1
>2022/06/29 13:46:53.352819 error UploadCompressedStream return error: 550 ftp_backup/shadow/test_db/test_table/default_all_3_3_0.tar: No such file or directory
>2022/06/29 13:46:53.356209 debug FTP::returnConnectionToPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_2_2_0.tar) active=1 idle=2
>2022/06/29 13:46:53.356545 debug FTP::PutFile key=ftp_backup/shadow/test_db/test_table/default_all_2_2_0.tar
>2022/06/29 13:46:53.356598 debug FTP::getConnectionFromPool(PutFile) active=0 idle=3
>2022/06/29 13:46:53.358011  warn FTP::MkdirAll MakeDir(ftp_backup) return error: 550 ftp_backup: Permission denied
>2022/06/29 13:46:53.358843  warn FTP::MkdirAll MakeDir(ftp_backup/shadow) return error: 550 ftp_backup/shadow: No such file or directory
>2022/06/29 13:46:53.359530  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db) return error: 550 ftp_backup/shadow/test_db: No such file or directory
>2022/06/29 13:46:53.360638  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db/test_table) return error: 550 ftp_backup/shadow/test_db/test_table: No such file or directory
>2022/06/29 13:46:53.362654 debug FTP::returnConnectionToPool(PutFile) active=1 idle=2
>2022/06/29 13:46:53.362792 error UploadCompressedStream return error: 550 ftp_backup/shadow/test_db/test_table/default_all_2_2_0.tar: No such file or directory
>2022/06/29 13:46:53.365016 debug FTP::returnConnectionToPool(StatFile, key=ftp_backup/shadow/test_db/test_table/default_all_1_1_0.tar) active=1 idle=3
>2022/06/29 13:46:53.365506 debug FTP::PutFile key=ftp_backup/shadow/test_db/test_table/default_all_1_1_0.tar
>2022/06/29 13:46:53.365618 debug FTP::getConnectionFromPool(PutFile) active=0 idle=4
>2022/06/29 13:46:53.366379  warn FTP::MkdirAll MakeDir(ftp_backup) return error: 550 ftp_backup: Permission denied
>2022/06/29 13:46:53.367627  warn FTP::MkdirAll MakeDir(ftp_backup/shadow) return error: 550 ftp_backup/shadow: No such file or directory
>2022/06/29 13:46:53.368135  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db) return error: 550 ftp_backup/shadow/test_db: No such file or directory
>2022/06/29 13:46:53.368642  warn FTP::MkdirAll MakeDir(ftp_backup/shadow/test_db/test_table) return error: 550 ftp_backup/shadow/test_db/test_table: No such file or directory
>2022/06/29 13:46:53.370719 debug FTP::returnConnectionToPool(PutFile) active=1 idle=3
>2022/06/29 13:46:53.370919 error UploadCompressedStream return error: 550 ftp_backup/shadow/test_db/test_table/default_all_1_1_0.tar: No such file or directory
>2022/06/29 13:46:53.371137 error one of upload go-routine return error: one of uploadTableData go-routine return error: can't upload: 550 ftp_backup/shadow/test_db/test_table/default_all_4_4_0.tar: No such file or directory

Actions: View

Attachments on bug 43104: 11005