What it is all about
The problem with FE-User passwords is not only that they are written in clear-text in the database and are by thus visible for BE-Editors at certain places (Even if you set 'eval' => 'password' the field will get shown as plain-text in the page-module when viewieng FE-Users on a page) but rather that the transmission between the clients browser and the server when a FE-User logs in is also in plain-text.
This enables malicous internet users which have the possibility to dump data which get's transfered from the client to the server to get the passwords of the FE-Users which log in about such an insecure connection.
When you would just md5 encode the password on the client-side and then send it over to the server a malicious internet user could still grab the md5-sum which get's sent along and use it to relogin as often as he wants.
To avoid such security leaks there exists a method which is called Challenge/Response. This method is also used when logging in into the TYPO3 backend by default. It is able to lower this security mechanism even to such a level that no java-script is required for BE login but this is absolutely not recommended and will not get described here.
How Challenge Response works
When a User wants to log in to a web-area secured with such a mechanism he first has to request theh login form. It is not possible to log in just by posting a specific value to a form - in every case you first have to request the login form to filter out the so called "Challenge" value from it.
md5pw = md5 ( password ); response = md5 ( username . md5pw . challenge );
It generates the response by building and md5-hash of the username concatenated with the md5-hash of the password and the challenge value extracted from the login-form which was requested first.
When the response get's sent to the server the server is performing the same calculation - it knows the md5-sum of the password from the value which is written in the database - the user which wants to log-in is sent along in the form - and the challenge can get re-associated to the client because of his session id.
Each challenge times out after a specific amount of time - and after every log-in. This removes the potential leak that a malicious user could log in by sending the same challenge/user/md5sum combination and it's md5-sum again. So each challenge can only be used once to log in and also times out after some time.
The above paragraph also explains why you can't log in into TYPO3 if you go to the login-page ... then let some time (about an half hour) expire and try to log-in afterwards. Because the challenge which was sent in the form is not valid any more.