Bug 22200
| Summary: | movemail can't download only new messages from IMAP | ||
|---|---|---|---|
| Product: | Sisyphus | Reporter: | Ivan Zakharyaschev <imz> |
| Component: | mailutils | Assignee: | Nobody's working on this, feel free to take it <nobody> |
| Status: | CLOSED WONTFIX | QA Contact: | qa-sisyphus |
| Severity: | enhancement | ||
| Priority: | P3 | ||
| Version: | unstable | ||
| Hardware: | all | ||
| OS: | Linux | ||
Can you describe this to bug-mailutils@gnu.org mail list ? I assembled a package mailutils only because this needed for mailfromd. (In reply to comment #1) > Can you describe this to bug-mailutils@gnu.org mail list ? I assembled a > package mailutils only because this needed for mailfromd. Thank you for your work! Perhaps, I will forward this feature request (or a "bug report", depending on whether they intended to have this feature) to the developers later, or any one else with more interest in the mailutils package should feel free to do so instead of me or you. I'd rather first explore the other known tools. Perhaps, storing UIDs is even a problem of the design of the Maildir format, and that makes it more difficult to implement what I wanted -- cf. /usr/share/doc/pine-4.64L/README.maildir: ---------- 5. On UIDs ---------- This patch does not keep UIDs between sessions, but hopefully it does keep consistent UIDs during one session. This is not a bug of the driver, instead it is a shortcoming of the maildir specification. The main point of the maildir configuration is that you should never (read my lips) ever edit the message, but edit the filename associated to the message. Well, I could not find any single place in the web where it was told how to save the UID of a message, if there is one please let me know and I will add UID support for this driver. (But there still seem to exist tools which work with Maildirs and that track UIDs: http://isync.sourceforge.net/ ; another one: http://mailsync.sourceforge.net/ , but this one tracks Message-ID fields instead.) (BTW, "mailutil transfer" from UW imap (pine-4.64L-alt6) doesn't help me: it also duplicates the messages when called like this: mailutil transfer -verbose -merge append '{arrakis/user=z/notls}a' '#driver.md/tmp/tests/mailing/z-mail_mailutil_transfer/' but that simply seems to conform to how it was intended to work.) and again propose to discuss this with the authors in http://lists.gnu.org/mailman/listinfo/bug-mailutils Please talk with autor in mail list "bug-mailutils". |
Description
Ivan Zakharyaschev 2009-11-07 11:46:53 MSKmailutils-2.1-alt0.20090928.3 I tried to mainatin a local (backup) copy of a remote IMAP mailbox with "movemail": obviously, I wanted that the second and furhther calls to "movemail" only transfer the new and updated messages, but in reality, "movemail" creates second copies of the already downloaded messages. "movemail" has an option with a semantics close to what I want; from "info mailutils": 2.6.1 Movemail Configuration ---------------------------- .... -- Movemail Config: uidl BOOL Avoid copying the message if a message with the same UIDL already exists in the destination mailbox. And some general info about "UIDL" from http://en.wikipedia.org/wiki/UIDL#Overview : Clients with a leave mail on server option generally use the POP3 UIDL (Unique IDentification Listing) command. Most POP3 commands identify specific messages by their ordinal number on the mail server. <...> UIDL provides a mechanism to avoid these numbering issues by assigning a string of characters as a permanent and unique ID for the message. <...> IMAP uses a 32-bit unique identifier (UID) that is assigned to messages in ascending (although not necessarily consecutive) order as they are received. When retrieving new messages, an IMAP client requests the UIDs greater than the highest UID among all previously-retrieved messages, whereas a POP client must fetch the entire UIDL map. For large mailboxes, this can require significant processing. So, in principle, a mechanism analoguous to UIDL is present in IMAP, and is even more effecient than its POP counterpart, so "movemail" could use it to download only new messages. But in practice, this is not so. A test case (the user "z" is a user for tests, the computer "arrakis" is running uw-imap-2001a-alt9.1): [ivan@dell ~]$ ssh arrakis Last login: Sat Nov 7 10:50:06 2009 from dell.zephyrous [ivan@arrakis data]$ sudo rm /var/mail/z [ivan@arrakis data]$ sudo touch /var/mail/z [ivan@arrakis data]$ sudo chown z /var/mail/z [ivan@arrakis data]$ l /var/mail/z -rw-r--r-- 1 z mail 0 Ноя 7 11:29 /var/mail/z [ivan@arrakis data]$ mail z Subject: test message A test Cc: [ivan@arrakis data]$ exit Connection to arrakis closed. [ivan@dell ~]$ movemail --verbose --keep --uidl 'imap://z@arrakis/INBOX' 'maildir:/home/ivan/tmp/z-mail-INBOX' Imap Passwd: movemail: info: number of messages in source mailbox: 1 movemail: info: number of processed messages: 1 [ivan@dell ~]$ l -R tmp/z-mail-INBOX/ tmp/z-mail-INBOX/: total 16 drwx------ 5 ivan ivan 144 Ноя 7 11:38 ./ drwxrwx--T 360 root ivan 12248 Ноя 7 11:38 ../ drwx------ 2 ivan ivan 48 Ноя 7 11:38 cur/ -rw-r--r-- 1 ivan ivan 3 Ноя 7 11:38 .mu-size drwx------ 2 ivan ivan 104 Ноя 7 11:38 new/ drwx------ 2 ivan ivan 48 Ноя 7 11:38 tmp/ tmp/z-mail-INBOX/cur: total 0 drwx------ 2 ivan ivan 48 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ tmp/z-mail-INBOX/new: total 4 drwx------ 2 ivan ivan 104 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ -rw------- 1 ivan ivan 561 Ноя 7 11:38 1257583083.M183312P16581Q0.dell.humboldt tmp/z-mail-INBOX/tmp: total 0 drwx------ 2 ivan ivan 48 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ [ivan@dell ~]$ movemail --verbose --keep --uidl 'imap://z@arrakis/INBOX' 'maildir:/home/ivan/tmp/z-mail-INBOX' Imap Passwd: movemail: info: number of messages in source mailbox: 1 movemail: info: number of processed messages: 1 [ivan@dell ~]$ l -R tmp/z-mail-INBOX/ tmp/z-mail-INBOX/: total 16 drwx------ 5 ivan ivan 144 Ноя 7 11:38 ./ drwx------ 2 ivan ivan 112 Ноя 7 11:38 cur/ -rw-r--r-- 1 ivan ivan 4 Ноя 7 11:38 .mu-size drwx------ 2 ivan ivan 104 Ноя 7 11:38 new/ drwx------ 2 ivan ivan 48 Ноя 7 11:38 tmp/ drwxrwx--T 360 root ivan 12248 Ноя 7 11:38 ../ tmp/z-mail-INBOX/cur: total 4 drwx------ 2 ivan ivan 112 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ -rw------- 1 ivan ivan 561 Ноя 7 11:38 1257583083.M183312P16581Q0.dell.humboldt:2 tmp/z-mail-INBOX/new: total 4 drwx------ 2 ivan ivan 104 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ -rw------- 1 ivan ivan 545 Ноя 7 11:38 1257583105.M518684P16583Q1.dell.humboldt tmp/z-mail-INBOX/tmp: total 0 drwx------ 2 ivan ivan 48 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ [ivan@dell ~]$ movemail --verbose --keep --uidl 'imap://z@arrakis/INBOX' 'maildir:/home/ivan/tmp/z-mail-INBOX' Imap Passwd: movemail: info: number of messages in source mailbox: 1 movemail: info: number of processed messages: 1 [ivan@dell ~]$ l -R tmp/z-mail-INBOX/ tmp/z-mail-INBOX/: total 16 drwx------ 5 ivan ivan 144 Ноя 7 11:38 ./ drwx------ 2 ivan ivan 176 Ноя 7 11:38 cur/ -rw-r--r-- 1 ivan ivan 4 Ноя 7 11:38 .mu-size drwx------ 2 ivan ivan 104 Ноя 7 11:38 new/ drwx------ 2 ivan ivan 48 Ноя 7 11:38 tmp/ drwxrwx--T 360 root ivan 12248 Ноя 7 11:38 ../ tmp/z-mail-INBOX/cur: total 8 drwx------ 2 ivan ivan 176 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ -rw------- 1 ivan ivan 561 Ноя 7 11:38 1257583083.M183312P16581Q0.dell.humboldt:2 -rw------- 1 ivan ivan 545 Ноя 7 11:38 1257583105.M518684P16583Q1.dell.humboldt:2 tmp/z-mail-INBOX/new: total 4 drwx------ 2 ivan ivan 104 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ -rw------- 1 ivan ivan 545 Ноя 7 11:38 1257583111.M478896P16585Q2.dell.humboldt tmp/z-mail-INBOX/tmp: total 0 drwx------ 2 ivan ivan 48 Ноя 7 11:38 ./ drwx------ 5 ivan ivan 144 Ноя 7 11:38 ../ [ivan@dell ~]$ So, the number of the copies of the message is growing. (Possible alternative solutions/tools for this task one could try: * "mailutil" from uw-imap or pine or alpiane -- http://www.washington.edu/imap/ ; * http://freshmeat.net/projects/imapsync ; * http://blog.hoopycat.com/2009/07/imap2maildir-a-tool-for-mirroring-imap-t , http://github.com/rtucker/imap2maildir ; * http://en.wikipedia.org/wiki/Getmail ; * fetchmail. )