18414 2008-12-31 20:35:43 +0300 rsyncd ignores "log format" 2009-01-01 20:33:48 +0300 1 1 4 Development Sisyphus rsync-server unstable all Linux CLOSED NOTABUG P2 normal --- 1 mike placeholder ab glebfm ldv placeholder vt qa-sisyphus oldest_to_newest 83720 0 mike 2008-12-31 20:35:43 +0300 # grep "^log format" /etc/rsyncd.conf log format = %a - - [%t] "GET /%m/%f" - %l "" "" Dec 31 19:13:02 pad rsyncd[15698]: connect from localhost.localdomain (127.0.0.1) Dec 31 19:13:02 pad rsyncd[15698]: rsync on pub/Linux/pmagic-3.1.iso.zip from localhost.localdomain (127.0.0.1) Dec 31 19:13:02 pad rsyncd[15698]: building file list Dec 31 19:13:03 pad rsyncd[15698]: sent 28298 bytes received 49353 bytes total size 49531904 Dec 31 19:23:52 pad rpm: rsync-2.6.3-alt3 installed Dec 31 19:23:53 pad rpm: rsync-3.0.4-alt3 removed Dec 31 19:23:53 pad rpm: rsync-server-2.6.3-alt3 installed Dec 31 19:23:53 pad rpm: rsync-server-3.0.4-alt3 removed Dec 31 19:24:13 pad rsyncd[16164]: rsync on pub/Linux/pmagic-3.1.iso.zip from localhost.localdomain (127.0.0.1) Dec 31 19:24:14 pad rsyncd[16164]: 127.0.0.1 - - [2008/12/31 17:24:14] "GET /pub/Linux/pmagic-3.1.iso.zip" - 49531904 "" "" Dec 31 19:24:14 pad rsyncd[16164]: wrote 28289 bytes read 49406 bytes total size 49531904 Thus this "log format" configuration worked for updates/Master/2.4 package but broke for Sisyphus one (it was noted that 2.6.9 would be broken already). Now it seems to be simply ignored. 83721 1 3176 mike 2008-12-31 20:38:56 +0300 Created attachment 3176 rsync2webalizer script PS: the whole affair was set up to make rsync generate CLF (common log format) logs to be fed into webalizer after being processed with the attached script by Jim Hefferon who wrote it for CTAN. 83757 2 ldv 2009-01-01 05:18:56 +0300 You forgot to enable "transfer logging". Starting with 2.6.4-alt1 (release date Mar 31 2005, see %changelog), this option is no longer enabled by default. 83785 3 mike 2009-01-01 20:07:00 +0300 Hm, I've definitely read up on that option and *seem* to have tried that... anyways, this works for me now with 3.0.4-alt3: log format = %a - - [%t] "GET /%m/%f" - %l "" "" transfer logging = yes syslog facility = ftp (just in case, "log file" adds stupid prefix to format string without being asked to) Sorry for bothering :) 83786 4 3180 mike 2009-01-01 20:33:47 +0300 Created attachment 3180 rsync2webalizer script OK, so here's an updated version of the script which would take current changes in logging into account (and works for me) -- just in case anyone needs to feed rsync stats into webalizer using a convertor to common log format. Should package it, probably... 3176 2008-12-31 20:38:56 +0300 2009-01-01 20:33:47 +0300 rsync2webalizer script rsync_log_massage.awk text/plain 2379 mike IyEvYmluL2F3awojICByc3luY19sb2dfbWFzc2FnZS5hd2sKIyBUd2lkZGxlIHRoZSByc3luYyBs b2cgZm9ybWF0IHRvIG1ha2UgaXQgd29yayBmb3IgdGhlIHdlYmFsaXplci4KIwojIFRoZSB3ZWJh bGl6ZXIgKHd3dy53ZWJhbGl6ZXIub3JnKSBhbmFseXplcyBsb2dzIGFuZCBwdXRzIHRoZSByZXN1 bHRzCiMgb250byBhIHBhZ2UuICBJdCBhc3NlcnRzIHRoYXQgaXQgc3VwcG9ydHMgY29tbW9uIGxv ZyBmb3JtYXQuICBUbyBnZXQKIyBpdCB0byBsaWtlIHRoZSBvdXRwdXQgb2YgbXkgcnN5bmMgSSBk aWQgdGhlIGZvbGxvd2luZy4KIyAgKDEpIEkgbWFkZSBteSByc3luYy5jb25mIGhhdmUgdGhlc2Ug bGluZXM6CiMgICAgIGxvZyBmaWxlID0gL3Zhci9sb2cvcnN5bmNkCiMgICAgIGxvZyBmb3JtYXQg PSAlYSAtIC0gIFsldF0gIkdFVCAlZiIgLSAlbCAiIiAiIgojICAoVGhlIGV4dHJhIHF1b3RlcyBh cmlzZSBmcm9tIHRoZSB3ZWJhbGl6ZXIgUkVBRE1FIGZpbGUgd2hpY2ggc2F5cwojICAgIFRoZSBX ZWJhbGl6ZXIgc3VwcG9ydHMgQ0xGIGxvZyBmb3JtYXRzLCB3aGljaCBzaG91bGQgd29yayBmb3Ig anVzdAojICAgIGFib3V0IGV2ZXJ5b25lLiAgSWYgeW91IHdhbnQgVXNlciBBZ2VudCBvciBSZWZl cnJlciBpbmZvcm1hdGlvbiwgeW91CiMgICAgbmVlZCB0byBtYWtlIHN1cmUgeW91ciB3ZWIgc2Vy dmVyIHN1cHBsaWVzIHRoaXMgaW5mb3JtYXRpb24gaW4gaXQncwojICAgIGxvZyBmaWxlLCBhbmQg aW4gYSBmb3JtYXQgdGhhdCB0aGUgV2ViYWxpemVyIGNhbiB1bmRlcnN0YW5kLiAgV2hpbGUKIyAg ICBUaGUgV2ViYWxpemVyIHdpbGwgdHJ5IHRvIGhhbmRsZSBtYW55IG9mIHRoZSBzdWJ0bGUgdmFy aWF0aW9ucyBpbgojICAgIGxvZyBmb3JtYXRzLCBzb21lIHdpbGwgbm90IHdvcmsgYXQgYWxsLiAg IE1vc3Qgd2ViIHNlcnZlcnMgb3V0cHV0CiMgICAgQ0xGIGZvcm1hdCBsb2dzIGJ5IGRlZmF1bHQu ICBGb3IgQXBhY2hlLCBpbiBvcmRlciB0byBwcm9kdWNlIHRoZQojICAgIHByb3BlciBsb2cgZm9y bWF0LCBhZGQgdGhlIGZvbGxvd2luZyB0byB0aGUgaHR0cGQuY29uZiBmaWxlOgojCiMgICAgTG9n Rm9ybWF0ICIlaCAlbCAldSAldCBcIiVyXCIgJXMgJWIgXCIle1JlZmVyZXJ9aVwiIFwiJXtVc2Vy LWFnZW50fWlcIiIKIwojICAgIFRoaXMgaW5zdHJ1Y3RzIHRoZSBBcGFjaGUgd2ViIHNlcnZlciB0 byBwcm9kdWNlIGEgJ2NvbWJpbmVkJyBsb2cKIyAgICB0aGF0IGluY2x1ZGVzIHRoZSByZWZlcnJl ciBhbmQgdXNlciBhZ2VudCBpbmZvcm1hdGlvbiBvbiB0aGUgZW5kIG9mCiMgICAgZWFjaCByZWNv cmQsIGVuY2xvc2VkIGluIHF1b3RlcyAoVGhpcyBpcyB0aGUgc3RhbmRhcmQgcmVjb21tZW5kZWQK IyAgICBieSBib3RoIEFwYWNoZSBhbmQgTkNTQSkuCiMgIC4gIEkgY291bGRuJ3QgZ2V0IHdlYmFs aXplciB0byBzaHV0IHVwIGFib3V0IGB0cnVuY2F0aW5nIHRoZSBsb25nIHJlZmVycmVyCiMgIGZp ZWxkJyBvdGhlcndpc2UuKQojICgyKSBJIGluc3RhbGxlZCB0aGlzIGF3ayBzY3JpcHQgdG8gcnVu IGFmdGVyIHRoZSBsb2dzIGFyZSByb3RhdGVkLgojICgzKSBJIHNldCB3ZWJhbGl6ZXIgdG8gcnVu IHdlZWtseSBvbiB0aGUgcG9zdC1tYXNzYWdlZCBsb2cgZmlsZS4KIwojIDIwMDItTWFyLTAxIEpp bSBIZWZmZXJvbiBmdHBtYWludEB0dWcuY3Rhbi5vcmcKIwpCRUdJTiB7bW9udGhzWzFdPSJKYW4i CiAgICAgICBtb250aHNbMl09IkZlYiIKICAgICAgIG1vbnRoc1szXT0iTWFyIgogICAgICAgbW9u dGhzWzRdPSJBcHIiCiAgICAgICBtb250aHNbNV09Ik1heSIKICAgICAgIG1vbnRoc1s2XT0iSnVu IgogICAgICAgbW9udGhzWzddPSJKdWwiCiAgICAgICBtb250aHNbOF09IkF1ZyIKICAgICAgIG1v bnRoc1s5XT0iU2VwIgogICAgICAgbW9udGhzWzEwXT0iT2N0IgogICAgICAgbW9udGhzWzExXT0i Tm92IgogICAgICAgbW9udGhzWzEyXT0iRGVjIn0KCiQ2fi9bMC05XStcLlswLTldK1wuWzAtOV0r XC5bMC05XSsvIHsKICAgIHllYXI9c3Vic3RyKCQ5LDIsNCk7CiAgICBtb250aD1zdWJzdHIoJDks NywyKTsKICAgIGRheT1zdWJzdHIoJDksMTAsMik7CiAgICBob3VyPXN1YnN0cigkMTAsMSwyKTsK ICAgIG1pbnM9c3Vic3RyKCQxMCw0LDIpOwogICAgc2Vjcz1zdWJzdHIoJDEwLDcsMik7CiAgICBt b249bW9udGhzW21vbnRoKzBdOwogICAgZm9yIChpPTY7IGk8PTg7IGkrKykgcHJpbnRmKCIlcyAi LCRpKTsKICAgIHByaW50ZigiIFslcy8lcy8lczolczolczolcyArMDAwMF0gIixkYXksbW9uLHll YXIsaG91cixtaW5zLHNlY3MpOwogICAgcHJpbnRmKCIiKQogICAgZm9yIChpPTExOyBpPD1ORjsg aSsrKSBwcmludGYoIiVzICIsJGkpOwogICAgcHJpbnRmKCJcbiIpCn0K 3180 2009-01-01 20:33:47 +0300 2009-01-01 20:33:47 +0300 rsync2webalizer script rsync_log_massage.awk application/octet-stream 3035 mike IyEvYmluL2F3awojICByc3luY19sb2dfbWFzc2FnZS5hd2sKIyBUd2lkZGxlIHRoZSByc3luYyBs b2cgZm9ybWF0IHRvIG1ha2UgaXQgd29yayBmb3IgdGhlIHdlYmFsaXplci4KIwojIFRoZSB3ZWJh bGl6ZXIgKHd3dy53ZWJhbGl6ZXIub3JnKSBhbmFseXplcyBsb2dzIGFuZCBwdXRzIHRoZSByZXN1 bHRzCiMgb250byBhIHBhZ2UuICBJdCBhc3NlcnRzIHRoYXQgaXQgc3VwcG9ydHMgY29tbW9uIGxv ZyBmb3JtYXQuICBUbyBnZXQKIyBpdCB0byBsaWtlIHRoZSBvdXRwdXQgb2YgbXkgcnN5bmMgSSBk aWQgdGhlIGZvbGxvd2luZy4KIyAgKDEpIEkgbWFkZSBteSByc3luYy5jb25mIGhhdmUgdGhlc2Ug bGluZXM6CiMgICAgIGxvZyBmaWxlID0gL3Zhci9sb2cvcnN5bmNkCiMgICAgIGxvZyBmb3JtYXQg PSAlYSAtIC0gIFsldF0gIkdFVCAlZiIgLSAlbCAiIiAiIgojICAoVGhlIGV4dHJhIHF1b3RlcyBh cmlzZSBmcm9tIHRoZSB3ZWJhbGl6ZXIgUkVBRE1FIGZpbGUgd2hpY2ggc2F5cwojICAgIFRoZSBX ZWJhbGl6ZXIgc3VwcG9ydHMgQ0xGIGxvZyBmb3JtYXRzLCB3aGljaCBzaG91bGQgd29yayBmb3Ig anVzdAojICAgIGFib3V0IGV2ZXJ5b25lLiAgSWYgeW91IHdhbnQgVXNlciBBZ2VudCBvciBSZWZl cnJlciBpbmZvcm1hdGlvbiwgeW91CiMgICAgbmVlZCB0byBtYWtlIHN1cmUgeW91ciB3ZWIgc2Vy dmVyIHN1cHBsaWVzIHRoaXMgaW5mb3JtYXRpb24gaW4gaXQncwojICAgIGxvZyBmaWxlLCBhbmQg aW4gYSBmb3JtYXQgdGhhdCB0aGUgV2ViYWxpemVyIGNhbiB1bmRlcnN0YW5kLiAgV2hpbGUKIyAg ICBUaGUgV2ViYWxpemVyIHdpbGwgdHJ5IHRvIGhhbmRsZSBtYW55IG9mIHRoZSBzdWJ0bGUgdmFy aWF0aW9ucyBpbgojICAgIGxvZyBmb3JtYXRzLCBzb21lIHdpbGwgbm90IHdvcmsgYXQgYWxsLiAg IE1vc3Qgd2ViIHNlcnZlcnMgb3V0cHV0CiMgICAgQ0xGIGZvcm1hdCBsb2dzIGJ5IGRlZmF1bHQu ICBGb3IgQXBhY2hlLCBpbiBvcmRlciB0byBwcm9kdWNlIHRoZQojICAgIHByb3BlciBsb2cgZm9y bWF0LCBhZGQgdGhlIGZvbGxvd2luZyB0byB0aGUgaHR0cGQuY29uZiBmaWxlOgojCiMgICAgTG9n Rm9ybWF0ICIlaCAlbCAldSAldCBcIiVyXCIgJXMgJWIgXCIle1JlZmVyZXJ9aVwiIFwiJXtVc2Vy LWFnZW50fWlcIiIKIwojICAgIFRoaXMgaW5zdHJ1Y3RzIHRoZSBBcGFjaGUgd2ViIHNlcnZlciB0 byBwcm9kdWNlIGEgJ2NvbWJpbmVkJyBsb2cKIyAgICB0aGF0IGluY2x1ZGVzIHRoZSByZWZlcnJl ciBhbmQgdXNlciBhZ2VudCBpbmZvcm1hdGlvbiBvbiB0aGUgZW5kIG9mCiMgICAgZWFjaCByZWNv cmQsIGVuY2xvc2VkIGluIHF1b3RlcyAoVGhpcyBpcyB0aGUgc3RhbmRhcmQgcmVjb21tZW5kZWQK IyAgICBieSBib3RoIEFwYWNoZSBhbmQgTkNTQSkuCiMgIC4gIEkgY291bGRuJ3QgZ2V0IHdlYmFs aXplciB0byBzaHV0IHVwIGFib3V0IGB0cnVuY2F0aW5nIHRoZSBsb25nIHJlZmVycmVyCiMgIGZp ZWxkJyBvdGhlcndpc2UuKQojICgyKSBJIGluc3RhbGxlZCB0aGlzIGF3ayBzY3JpcHQgdG8gcnVu IGFmdGVyIHRoZSBsb2dzIGFyZSByb3RhdGVkLgojICgzKSBJIHNldCB3ZWJhbGl6ZXIgdG8gcnVu IHdlZWtseSBvbiB0aGUgcG9zdC1tYXNzYWdlZCBsb2cgZmlsZS4KIwojIDIwMDItTWFyLTAxIEpp bSBIZWZmZXJvbiBmdHBtYWludEB0dWcuY3Rhbi5vcmcKIwoKIyBxdWljayBob3d0byBvbiBnZXR0 aW5nIHRoaW5ncyB0b2dldGhlciAoTWljaGFlbCBTaGlnb3Jpbik6CiMgLS0tIHJzeW5jZC5jb25m CiMgc3lzbG9nIGZhY2lsaXR5ID0gZnRwCiMgdHJhbnNmZXIgbG9nZ2luZyA9IHllcwojIGxvZyBm b3JtYXQgPSAlYSAtIC0gIFsldF0gIkdFVCAvJW0vJWYiIC0gJWwgIiIgIiIKIyAtLS0gL3Vzci9s b2NhbC9zYmluL3JzeW5jMndlYmFsaXplci5zaCAKIyAjIS9iaW4vc2gKIyB1bWFzayAwNzcKIyBu aWNlIHRpbWUgYXdrIC1mIC91c3IvbG9jYWwvYmluL3JzeW5jX2xvZ19tYXNzYWdlLmF3ayAvdmFy L2xvZy9yc3luY2QubG9nID4gL3Zhci9sb2cvcnN5bmNkLmxvZy5DTEYgXAojICYmIGNob3duIHdl YmFsaXplciAvdmFyL2xvZy9yc3luY2QubG9nLkNMRiBcCiMgJiYgc3UgLSAtYyAibmljZSB0aW1l IHdlYmFsaXplciAtYyAvZXRjL3dlYmFsaXplci93ZWJhbGl6ZXItcnN5bmMuY29uZiIgLXMgL2Jp bi9zaCB3ZWJhbGl6ZXIgXAojICYmIDo+IC92YXIvbG9nL3JzeW5jZC5sb2cuQ0xGIFwKIyAmJiA6 Pi92YXIvbG9nL3JzeW5jZC5sb2cKIyAtLS0gZW5qb3kgOy0pCgpCRUdJTiB7bW9udGhzWzFdPSJK YW4iCiAgICAgICBtb250aHNbMl09IkZlYiIKICAgICAgIG1vbnRoc1szXT0iTWFyIgogICAgICAg bW9udGhzWzRdPSJBcHIiCiAgICAgICBtb250aHNbNV09Ik1heSIKICAgICAgIG1vbnRoc1s2XT0i SnVuIgogICAgICAgbW9udGhzWzddPSJKdWwiCiAgICAgICBtb250aHNbOF09IkF1ZyIKICAgICAg IG1vbnRoc1s5XT0iU2VwIgogICAgICAgbW9udGhzWzEwXT0iT2N0IgogICAgICAgbW9udGhzWzEx XT0iTm92IgogICAgICAgbW9udGhzWzEyXT0iRGVjIn0KCgkgICAkNn4vWzAtOV0rXC5bMC05XStc LlswLTldK1wuWzAtOV0rLyB7CgkJCSAgIGlmICgkNyA9PSAiLSIpIHsKCQkJCQkgICB5ZWFyPXN1 YnN0cigkOSwyLDQpOwoJCQkJCSAgIG1vbnRoPXN1YnN0cigkOSw3LDIpOwoJCQkJCSAgIGRheT1z dWJzdHIoJDksMTAsMik7CgkJCQkJICAgaG91cj1zdWJzdHIoJDEwLDEsMik7CgkJCQkJICAgbWlu cz1zdWJzdHIoJDEwLDQsMik7CgkJCQkJICAgc2Vjcz1zdWJzdHIoJDEwLDcsMik7CgkJCQkJICAg bW9uPW1vbnRoc1ttb250aCswXTsKCQkJCQkgICBmb3IgKGk9NjsgaTw9ODsgaSsrKSBwcmludGYo IiVzICIsJGkpOwoJCQkJCSAgIHByaW50ZigiIFslcy8lcy8lczolczolczolcyArMDAwMF0gIixk YXksbW9uLHllYXIsaG91cixtaW5zLHNlY3MpOwoJCQkJCSAgIHByaW50ZigiIikKCQkJCQkgICBm b3IgKGk9MTE7IGk8PU5GOyBpKyspIHByaW50ZigiJXMgIiwkaSk7CgkJCQkJICAgcHJpbnRmKCJc biIpCgkJCSAgIH0KfQo=