Problem with XMail, POP and all Webmail Software

[dolem98]

I have discovered a problem when using Xmail with POP and Webmail.

I have had this problem using different webmail software, including:
-Horde
-Ilohamail
-umpl_xmail
-dwmail

Since the problem occurs in so many different types of software, it may be something with Xmail, and it may be something with POP.

The Problem: When you click to view an email, from your webmail inbox, the wrong email is shown. When you click to delete an email, from your webmail inbox, the wrong email is deleted.

How to generate the problem:
-Have a few emails in your inbox, log into your webmail and list out your inbox.
-From another window, or Outlook, or somewhere else, send yourself an email.
-Wait a minute or so to let your server process the email.
-Click on an email in your inbox. If the wrong email shows up, you have created the same error as me.
OR, delete an email in your inbox. If the wrong email is deleted you have experienced the same problem.

What causes the problem:
It seems like an index problem. Lets say you have three emails in your inbox. Each email will have a link on it so that you can click in to view that email, something like this:
My Inbox
view.php?id=1 Email From Bob
view.php?id=2 Email From Mary
view.php?id=3 Email From Ted

Now pretend that as you are viewing your inbox, Jill sends you an email. Before you refresh your inbox, you decide to click on the Email From Mary to view it.

Instead, you will be seeing the email from Bob. How come? It's because the indexes have shifted. If you go back to your inbox, it will be something like this:
My Inbox
view.php?id=1 Email From Jill
view.php?id=2 Email From Bob
view.php?id=3 Email From Mary
view.php?id=4 Email From Ted

Now you see why clicking on the email with an index of 2 brought up Bob's email rather than Mary's. This is a way oversimplified version of the problem, as the indexes are generally not in order (but they can be).



Has anyone else noticed anything similar, or am I alone out there? Any suggestions?

[jobber_jobber]

Hi,

This sounded most worrying, so tried it out, using:

Xmail v1.21
IlohaMail v0.8.14-RC2

Sorry, but can't reproduce the problem either with viewing mails or deleting mails.

Regards,

Jobber

[hschneider]

Hi,

I experienced the same issue with Ilohamail a few weeks ago and was able to track down this error. It is definately NO XMail related bug!

The index gets confused when you e.g. delete mails and use your browser's back button. This generates a HTML post event from an expired form, which holds the wrong index number in the delete command. That bug is buried in the PHP routine which reads the POP box respectively in the software's design concept itself. I'm pretty sure that most webmail solutions use the same buggy basic routine or index scheme.

I've posted this to the Ilohamail mailing list, but there seems to be no reaction from the author. All you can do is to report this issue to the author, that this bug gets hopefully recognized.

As a workaround, I recommend to run webmail in a window, which has the browsers forward and back buttons disabled.. But this is no 100% solution.

[dolem98]

jobber, are you sure you tested it right? I just downloaded the latest version of Ilohamail and I could recreate the bug.

I have an idea for a solution, given by Francis Clement on the xmail discussion list. Apparently you can use the POP command UIDL instead of LIST and it supplies a unique identifier. My goal is going to be to rewrite one of the webmail solutions to take advantage of this.

[hschneider]

That's right - using UIDL is the only way to avoid such error. But patching an existing solution won't be trivial because the index is used in many URL's and functions. Even the DELE and RETR commands require an index number instead of an UID. I think this is the cause, why all existing webmail solutions own that bug.

So what you will need is a function which returns the current index of a given UID and vice versa. But keep in mind, that the assignment of an index to an UID also changes with each DELE operation ...

[dolem98]

I am amazed that no pop3 webmail software is written with UIDL in mind. It is a huge problem... we have lots of clients who will go in and delete an email and it will end up deleting the wrong one.

I really do not want to have to support an IMAP Server (too much disk space) so I am trying to think of another route.

[hschneider]

This index shifting is simply a thing which is easily overseen during a program's testing phase. The root of all this is the POP3 protocol itself, which was never designed for selective message deletion before polling a POP box.

[dolem98]

So the solution is...

Use IMAP

or

Develop webmail software which interacts with Xmail's directory structure. (Does anything like this exist?)

[hschneider]

No - I think the problem is, that Xmail runs as root which collides with most webserver setups. Using UIDL is a good starting point to fix existing code or to create new one ...

[dolem98]

A Horde developer claims that it uses UIDL for POP. The id's are just a pointer to a UIDL stored internally. However it has the same inconsistency issue. Is it possible that xmail does not properly report UIDLs?

[hschneider]

If it would, you would have the same issue with your mail client.

According to horde:
Let's assume you have 2 parallel arrays which link the pointers to the UIDL's internally:

1 -> X1
2 -> X2
3 -> X3
4 -> X4

Now we delete ID 2:

1 -> X1
2 -> X3
3 -> X4

It is very likely that this shifting happens, because after a DELE command horde will read the POP account again in order to scan for new messages, doing a renumbering of the left array.

Now the user uses the back button and reposts a command which deletes ID 2 (e.g. something.php?cmd=delete&id=2 ) ... the result: X3 is gone !

1 -> X1
2 -> X4

With other words:
The ID coming with the delete request, just describes the position in a list.
If the list changed meanwhile, the wrong candidate will be deleted. Problem: the list changes after each refresh, when the POP box is read again.

As long as a UIDL is NOT submitted with the URL, this bug is still present.
Another solution would be to send a timestamp with each URL. So the application can determine, if a request has been expired.

Feel free to forward this to the horde or any other webmailer list.