Project

General

Profile

Bug #392

Writes invalid XML chars to the database

Added by Sander Steenhuis almost 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
High
Assignee:
-
Target version:
Operating System:
Linux
Affected version:
Affected revision:

Description

I imported my KeePassX 0.4 database in KeePassX 2.0 (2.0.0-1~getdeb1) on Linux Mint 17.3. Saved it. Opened it to a different computer. Can't open:

Unable to open the database.
XML error:
Unexpected ''.
Line 30283, column 19

This file was imported and saved by KeePassX 2.0 itself, and crucial. For now I'm downgrading to KeePassX 0.4.

test-database.kdbx (1.58 KB) Clinton Begin, 12/21/2015 01:23 AM

Associated revisions

Revision 2d741afe
Added by Felix Geyer over 1 year ago

Strip invalid XML chars when writing databases.

These characters are unprintable or just plain invalid.
QXmlStreamReader throws and error when reading XML documents with such chars.

Closes #392

Revision eb56bd89
Added by Felix Geyer over 1 year ago

Add repair functionality to strip invalid XML chars.

Refs #392

History

#1 Updated by Felix Geyer almost 2 years ago

Can you still open the database on the machine you imported it?
Can you reproduce this with a new KeePassX 0.4 database?

#2 Updated by Sander Steenhuis almost 2 years ago

No, the machine that saved the file also cannot open it.

It's not a faulty storage. When I re-import it and save it, it fails to open again with the same error.

I can however NOT reproduce this with a new KeePassX 0.4 database. So I think something in my original database cannot be stored and read back in the 2.0 format.

However, I cannot share my database for obvious reasons. If there is a way for me to see the decoded XML I'd be happy to look at the position mentioned by the error message.

#3 Updated by Felix Geyer almost 2 years ago

When you build KeePassX from source you get the command line binary utils/kdbx-extract in your build dir.
It outputs the XML that corresponds to the error message.

#4 Updated by Sander Steenhuis almost 2 years ago

That's perhaps a bit too complex for me. I'm a node dev, everything compiles automatically and just-in-time. I'd need to take a few courses on low level languages and manual compiling first.

Any chance you can upload or send me the binary, or provide a link to a package that contains the binary?

I'm on X64 (Linux 3.19.0-32-generic #37~14.04.1-Ubuntu SMP x86_64 GNU/Linux)

#5 Updated by Clinton Begin almost 2 years ago

Hi guys,

I had this error too, and decided to spend a few hours narrowing down the cause. I've depended on KeePassX for many years, so it's the least I could do. The problem can be recreated very simply. In the test case that I use to recreate it, the problem seems to be caused by some disagreeable data in the Notes field of an entry.

In my case, I had pasted an IRC instruction block into the comments. To recreate a broken database:

Step 1. Create a new, empty database. Any password can be used. I just used 'test'.

Step 2. Add a new entry in the Root group.

Step 3. I left all other fields blank, but I don't think it mattes. You can fill them in if you like.

Step 4. In the 'Notes' section, copy and paste the following block. I (hopefully) scrubbed all of my personal data out of it. DO NOT include the opening and closing hyphenated lines.


-------
NickServ NOTICE An email containing nickname activation instructions has been sent to xxxxxxx.xxxxx@xxxxx.xxx.
NickServ NOTICE If you do not complete registration within one day, your nickname will expire.
NickServ NOTICE xxxxxxxxx is now registered to xxxxxxx.xxxxx@xxxxx.xxx, with the password xxxxxxxx.
NickServ NOTICE
NickServ NOTICE For frequently-asked questions about the network, please see the
NickServ NOTICE FAQ page (http://freenode.net/faq.shtml). Should you need more
NickServ NOTICE help you can /join #freenode to find network staff.
-------

Step 5. Save the database. Close the database. Reopen the database. (don't just reopen it without closing it first, as KeePassX believes it's already open).

You will get the XML error as described.

Cheers!

#6 Updated by Clinton Begin almost 2 years ago

I've attached a bad database file that contains only test data to hopefully help speed you along.

#7 Updated by Sander Steenhuis almost 2 years ago

That's awesome, Clinton Begin. Thanks for figuring that out. I use comments/notes/description extensively in my 0.4 database. It never crossed my mind that herein might be the problem.

Especially the lack of 'common error' prone characters in your note/comment is puzzling.

#8 Updated by Sander Steenhuis almost 2 years ago

For the KeePassX team, I would like to point out that it's unsettling that a wrong comment could save/overwrite the database in a broken state without any alarm bells. Only when opening it another day does trouble show itself, and it will be too late.

Perhaps a safety measurement to prevent unforeseen consequences is an idea. Like saving the database to a temporary file, opening it under the hood to see if it can be opened, and only when everything works out, replace the database that is saved.

#9 Updated by Felix Geyer almost 2 years ago

  • Target version set to 2.0.1

Clinton Begin wrote:

I've attached a bad database file that contains only test data to hopefully help speed you along.

Thanks for tracking this down!

Seems like the Qt XML classes happily write non-printable chars (0x02 in this case) but trip over when parsing them.

#10 Updated by Georg Weber over 1 year ago

Unable to open the database. XML error: Unexpected ". Line 119, column 20 is the response to this text in the Notes window:
HKEY_CURRENT_USER\Software\Microsoft\NetDDE\DDE Trusted Shares\D06D239C0\Chat$
Serial Number: ?HKEY_CURRENT_USER\Software\Microsoft\NetDDE\DDE Trusted Shares\D06D239C0\CLPBK$
Serial Number: ?HKEY_CURRENT_USER\Software\Microsoft\NetDDE\DDE Trusted Shares\D06D239C0\Hearts$
Serial Number: ?HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Registration
Product ID: 76497-OEM-0052151-48208
Product Type: OEM Version

OS: OSX 10.10, KeePassX 2.0, Qt 4.8.7, libgcrypt 1.6.4

#11 Updated by Karma Kollabor over 1 year ago

This error hit me too and after being a little bit too naive reading your announcement about "first stable release of the KeePassX 2 series after several years of development" this problem triggered a near catastrophic event.

Please immediately stop distributing this as "stable" or at least put a BIG RED WARNING on the download page mentioning this error that WILL DESTROY YOUR KEEPASS FILE and disable access to ALL YOUR PASSWORDS. Yes, of course I have backups, but when this happens after a few months and all my backups are Version 2 files and all of them are broken in this way then the world will be a very dark place. This is a predictable scenario for many people right now so the distribution should be stopped immediately to avoid future mass suicides of keepassx version 2 users.

I would like to know (this is a serious research question, no trolling): how could such a destructive error occur while this project having lots of test cases? I did not study the tests in the source, but it looks like test coverage seems to be not so bad. What happened (or did not happen) that such a huge problem could survive for many years in this codebase? Any hints?

Please do not take this as "blaming" or "shaming" - I know too well that errors in software development always happen, I am just interested in why so many projects that are using software testing still have catastrophic bugs. You might guess that I tend to believe that testing seems to be an occult myth - but I am not quite finished with my research on this, so there is no final judgement yet. But there are too many projects where testing seems to give a false perception of everything being ok with still destructive errors in the code.

Thanks for your attention.

#12 Updated by Karma Kollabor over 1 year ago

I just send an email to the maintainer of the getdeb.org repo to take that broken version offline until it is fixed.

If you know any other sources that distribute this broken version of keepassx you might use this as a template:

Hi, {$name},

as you can see in this bug https://www.keepassx.org/dev/issues/392

keepassx 2 has a catastrophic error that destroys the database - it is clearly predictable that after a few months of usage, when this happens, there will be many users that can not open their keepass file anymore - if their backups rotated long enough and they have only keepass 2 backups that will lead to a situation where it is quite probable that they will loose all their passwords.

To avoid that, it would be important to stop distribution of keepassx immediately until this error is fixed.
Please help to protect keepassx version 2 users by disabling the package in your repos until a fix is available, thank you!

#13 Updated by Jens Yllman over 1 year ago

I have keepassx 2.0 both from debian sid repo using QT 4.8.7 and a version build from latest github using QT 5,5,1, and both crashes. Is it verified that this is a QT problem? And in that case is it reported as a bug to QT?

#14 Updated by Florian Geyer over 1 year ago

If you are affected by this issue and don't have a backup to revert to, do not delete your database.
We'll provide some kind of repair functionality in the next version which strips out the invalid chars.

#15 Updated by Felix Geyer over 1 year ago

  • Subject changed from XML error to Writes invalid XML chars to the database

#16 Updated by Felix Geyer over 1 year ago

  • Status changed from New to Closed

Also available in: Atom PDF