Writes invalid XML chars to the database
I imported my KeePassX 0.4 database in KeePassX 2.0 (2.0.0-1~getdeb1) on Linux Mint 17.3. Saved it. Opened it to a different computer. Can't open:
Unable to open the database.
Line 30283, column 19
This file was imported and saved by KeePassX 2.0 itself, and crucial. For now I'm downgrading to KeePassX 0.4.
Strip invalid XML chars when writing databases.
These characters are unprintable or just plain invalid.
QXmlStreamReader throws and error when reading XML documents with such chars.
#2 Updated by Sander Steenhuis over 3 years ago
No, the machine that saved the file also cannot open it.
It's not a faulty storage. When I re-import it and save it, it fails to open again with the same error.
I can however NOT reproduce this with a new KeePassX 0.4 database. So I think something in my original database cannot be stored and read back in the 2.0 format.
However, I cannot share my database for obvious reasons. If there is a way for me to see the decoded XML I'd be happy to look at the position mentioned by the error message.
#4 Updated by Sander Steenhuis over 3 years ago
That's perhaps a bit too complex for me. I'm a node dev, everything compiles automatically and just-in-time. I'd need to take a few courses on low level languages and manual compiling first.
Any chance you can upload or send me the binary, or provide a link to a package that contains the binary?
I'm on X64 (Linux 3.19.0-32-generic #37~14.04.1-Ubuntu SMP x86_64 GNU/Linux)
#5 Updated by Clinton Begin over 3 years ago
I had this error too, and decided to spend a few hours narrowing down the cause. I've depended on KeePassX for many years, so it's the least I could do. The problem can be recreated very simply. In the test case that I use to recreate it, the problem seems to be caused by some disagreeable data in the Notes field of an entry.
In my case, I had pasted an IRC instruction block into the comments. To recreate a broken database:
Step 1. Create a new, empty database. Any password can be used. I just used 'test'.
Step 2. Add a new entry in the Root group.
Step 3. I left all other fields blank, but I don't think it mattes. You can fill them in if you like.
Step 4. In the 'Notes' section, copy and paste the following block. I (hopefully) scrubbed all of my personal data out of it. DO NOT include the opening and closing hyphenated lines.
NickServ NOTICE An email containing nickname activation instructions has been sent to firstname.lastname@example.org.
NickServ NOTICE If you do not complete registration within one day, your nickname will expire.
NickServ NOTICE xxxxxxxxx is now registered to email@example.com, with the password xxxxxxxx.
NickServ NOTICE For frequently-asked questions about the network, please see the
NickServ NOTICE FAQ page (http://freenode.net/faq.shtml). Should you need more
NickServ NOTICE help you can /join #freenode to find network staff.
Step 5. Save the database. Close the database. Reopen the database. (don't just reopen it without closing it first, as KeePassX believes it's already open).
You will get the XML error as described.
#7 Updated by Sander Steenhuis over 3 years ago
That's awesome, Clinton Begin. Thanks for figuring that out. I use comments/notes/description extensively in my 0.4 database. It never crossed my mind that herein might be the problem.
Especially the lack of 'common error' prone characters in your note/comment is puzzling.
#8 Updated by Sander Steenhuis over 3 years ago
For the KeePassX team, I would like to point out that it's unsettling that a wrong comment could save/overwrite the database in a broken state without any alarm bells. Only when opening it another day does trouble show itself, and it will be too late.
Perhaps a safety measurement to prevent unforeseen consequences is an idea. Like saving the database to a temporary file, opening it under the hood to see if it can be opened, and only when everything works out, replace the database that is saved.
#9 Updated by Felix Geyer about 3 years ago
- Target version set to 2.0.1
Clinton Begin wrote:
I've attached a bad database file that contains only test data to hopefully help speed you along.
Thanks for tracking this down!
Seems like the Qt XML classes happily write non-printable chars (0x02 in this case) but trip over when parsing them.
#10 Updated by Georg Weber about 3 years ago
Unable to open the database. XML error: Unexpected ". Line 119, column 20 is the response to this text in the Notes window:
HKEY_CURRENT_USER\Software\Microsoft\NetDDE\DDE Trusted Shares\D06D239C0\Chat$
Serial Number: ?HKEY_CURRENT_USER\Software\Microsoft\NetDDE\DDE Trusted Shares\D06D239C0\CLPBK$
Serial Number: ?HKEY_CURRENT_USER\Software\Microsoft\NetDDE\DDE Trusted Shares\D06D239C0\Hearts$
Serial Number: ?HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Registration
Product ID: 76497-OEM-0052151-48208
Product Type: OEM Version
OS: OSX 10.10, KeePassX 2.0, Qt 4.8.7, libgcrypt 1.6.4
#11 Updated by Karma Kollabor about 3 years ago
This error hit me too and after being a little bit too naive reading your announcement about "first stable release of the KeePassX 2 series after several years of development" this problem triggered a near catastrophic event.
Please immediately stop distributing this as "stable" or at least put a BIG RED WARNING on the download page mentioning this error that WILL DESTROY YOUR KEEPASS FILE and disable access to ALL YOUR PASSWORDS. Yes, of course I have backups, but when this happens after a few months and all my backups are Version 2 files and all of them are broken in this way then the world will be a very dark place. This is a predictable scenario for many people right now so the distribution should be stopped immediately to avoid future mass suicides of keepassx version 2 users.
I would like to know (this is a serious research question, no trolling): how could such a destructive error occur while this project having lots of test cases? I did not study the tests in the source, but it looks like test coverage seems to be not so bad. What happened (or did not happen) that such a huge problem could survive for many years in this codebase? Any hints?
Please do not take this as "blaming" or "shaming" - I know too well that errors in software development always happen, I am just interested in why so many projects that are using software testing still have catastrophic bugs. You might guess that I tend to believe that testing seems to be an occult myth - but I am not quite finished with my research on this, so there is no final judgement yet. But there are too many projects where testing seems to give a false perception of everything being ok with still destructive errors in the code.
Thanks for your attention.
#12 Updated by Karma Kollabor about 3 years ago
I just send an email to the maintainer of the getdeb.org repo to take that broken version offline until it is fixed.
If you know any other sources that distribute this broken version of keepassx you might use this as a template:
as you can see in this bug https://www.keepassx.org/dev/issues/392
keepassx 2 has a catastrophic error that destroys the database - it is clearly predictable that after a few months of usage, when this happens, there will be many users that can not open their keepass file anymore - if their backups rotated long enough and they have only keepass 2 backups that will lead to a situation where it is quite probable that they will loose all their passwords.
To avoid that, it would be important to stop distribution of keepassx immediately until this error is fixed.
Please help to protect keepassx version 2 users by disabling the package in your repos until a fix is available, thank you!